Follow-up on resource reclamation: Search resources and Code Engine commands

Restore IBM Cloud Code Engine projects

Some longer I time ago, I wrote about resource reclamation in IBM Cloud. When you delete a resource, it is not immediately gone but moved into a kind of trash bin. It gives you some days to reconsider or to recover from accidents. Today, I am going to share some additional tips & tricks.

A simple nginx deployment on IBM Cloud Code Engine

HTML files as configmap in Code Engine

Over the weekend I deployed nginx on IBM Cloud Code Engine. It's nothing fancy or earth-moving. But I wanted to note down and share the steps with you on how to deploy this HTTP server and serve few static HTML pages. All is based on using the command line interface (CLI) and is done with few commands.

Custom domain for your serverless Code Engine app

Output of a IBM Cloud Code Engine app

A custom domain, e.g. somethin like myapp.example.com, always puts a nice touch on an app. Often, it is required for corporate identity. Still, not all services offer that capability. As written, I recently migrated some apps from Cloud Foundry to IBM Cloud Code Engine. Right now, Code Engine does not support custom domains out of the box, but I heard that it is planned. Thus, I looked into temporary workarounds for using a custom domain name for my serverless Code Engine apps.

Blue-green deployment with IBM Cloud Code Engine and Knative

Code Engine app with green revision

From reading my past post on migrating from Cloud Foundry to Code Engine, you probably guessed that I am currently working with IBM Cloud Code Engine. One of the topics I looked into the past few days is advanced configuration of my app using the Knative CLI. Code Engine is using Knative which, again, is based on Kubernetes. All of them are involved in configuring traffic management for my app, so that I could realize blue-green deployment. It allows to gradually move traffic from the current stable to a newer app revision. Not much is needed to get it to work.

Quickly deploy the serverless cloud mailer using Terraform

In the era of instant messaging we all still receive emails. They are used for status updates, security alerts or just for proposing really great offers. Recently, I blogged about how to have the IBM Cloud Security Advisor send out alerts using your SMTP-based email delivery service. Later, I made the solution core, a serverless action available as separate project "cloudmailer" on GitHub and blogged about it: A Serverless Function for Sending Emails on IBM Cloud. Continuing this side project, I now added Terraform support. Thus, using "terraform apply" you can now automatically deploy everything including the SMTP configuration. See the instructions in code repository for details.

IBM Cloud: Send out alert emails using SMTP

Last year, I blogged about how to use Slack or email notifications for security IBM Cloud security issues. Now I added another code sample, showing how to send out notification emails using any SMTP server. The code is written in Python and Node.js and deployed as IBM Cloud Functions-based webhook. The core part in charge of connecting to an SMTP email server and sending the email is isolated as dedicated action. Thus, it is possible to use it on its own, e.g., to send out emails unrelated to the IBM Cloud Security & Compliance Center.

cron-like scheduling on IBM Cloud

cron-like scheduling

Some days ago I stumbled over my 2015 post "Bluemix: Simple cron-like service for my Python code". It is not just the name Bluemix which is dated. Since then, it has transformed into IBM Cloud Platform and has added serverless compute options like IBM Cloud Functions (OpenWhisk) and recently IBM Cloud Code Engine. Both of them support "eventing", event-based execution of code. And both support Python code as well as many other programming languages like Node.js, Go (Golang), PHP and more. So, what does it take to set up cron-like scheduling? Not much.

All the best combined: Cloud, Db2, Python, Serverless and Security

Add a cloud service ID as Db2 user

Earlier this year, I wrote about how to use an API key or access token to connect to Db2 (on Cloud). Today, I am going to show you how to set up a service ID (technical user) on IBM Cloud, assign it a Db2 user ID (see screenshot) and grant database privileges. Thereafter, I share Python code for connecting to Db2 using the service ID with an API key.

All this helps to reduce the set of privileges held by a user or service and hence increases cloud security.

How to: Slack or email notifications for IBM Cloud security issues

Architecture for security notifications

In my past blogs I introduced you to IBM Cloud Security Advisor and how to integrate your own security scans. You can view all security findings in the security advisor dashboard. Another option is to get alerted for your choice of new security issues. For this you need to set up notification channels and messaging. In today's blog I am going to explain details and share my code.

Rotating service credentials for IBM Cloud Functions

Keep your service keys secret

If you have followed some of my work, you know that I use IBM Cloud Functions, i.e., a serverless approach, for many projects. The tutorials with a database-driven (Db2-backed) Slackbot and the GitHub traffic analytics are such examples. In this blog post, I want to detail some of the security-related aspects. This includes how to share service credentials (think of a database username and password) with a cloud function and how to rotate the credentials.

Automated reports with IBM Cloud Functions, Db2 and Slack

GitHub Traffic Analytics

One of my (many) favorite IBM Cloud solution tutorials is about combining serverless and Cloud Foundry for data retrieval and analytics. I blogged about it and described how an automated IBM Cloud Functions action retrieves GitHub statistics and stores them in Db2. Using an embedded Cognos dashboard and regular Javascript / HTML tables, the solution offers GitHub Traffic Analytics. I extended that solution by automatic weekly reports that are posted to Slack.

Extended: Manage and interact with Watson Assistant from the command line

Remember my blog posts about how to manage Watson Assistant from the command line and how to test context for a conversation? Well, that tool did not work well for server actions which I used in this tutorial on building database-driven Slackbot. The good news is that I found time to extend my command line Watson Conversation Tool to support credentials for IBM Cloud Functions.

With the recent update to the tool there are two new features:

1. Use the option "-outputonly" with the "-dialog" option to only print the output text, not the entire JSON response object. I introduced it to be able to demo dialog flows from the command line. Not everybody needs all the metadata for every dialog turn. Here is how it looks like when in action:



Chatbot dialog on the command line

2. In order to test dialog server actions, I need to provide the credentials for IBM Cloud Functions (ICF) in a private context variable. I recently blogged about how to enable the Watson botkit middleware for those server actions. For my tool, just provide the ICF key token as part of the configuration file. A sample is part of the GitHub repository.

For feature requests, defects or suggestions, open an issue or a pull request against the repository. If you have feedback, suggestions, or questions about this post, please reach out to me on Twitter (@data_henrik) or LinkedIn.

How to pack serverless Python actions

Serverless access to Db2 and GitHub

For my tutorial on automated data retrieval and analytics, I use IBM Cloud Functions to automatically fetch GitHub traffic statistics once a day. It is implemented as a serverless Python action. Because some Python packages are needed, the question was how to pack and create the action. In this blog, I share some of my experiences.

Use Db2 and IBM Cloud to analyze GitHub traffic data (tutorial)

Architecture: GitHub Traffic Analytics

In a new solution tutorial, I show you how to automatically retrieve and store GitHub traffic data the serverless way with IBM Cloud Functions and Db2. The data can then be analyzed via a Web app deployed to Cloud Foundry on IBM Cloud. The app is secured with App ID using OpenID Connect. The new service Dynamic Dashboard Embedded provides visualization of the views and clones of GitHub repositories.

Security Details: Serverless database access within IBM Watson Conversation service

Slackbot Architecture

Last week, I introduced you to a new tutorial for a database-driven Slackbot. Today, I am going to discuss details of how the IBM Watson Conversation service is accessing a Db2 Warehouse service from within a dialog. It uses a serverless setup with IBM Cloud Functions. All the necessary credentials to execute the code and to access the Db2 database are automatically bound. Hence, the function code and the dialog don't need any account-specific changes and are generic.

Easy Database Setup the Serverless Way

Serverless Slackbot with Db2

A tutorial I wrote, featuring a database-backed Slack chatbot, is now live. It uses Db2 as database system to store event data. The client accessing the database is written in Node.js and is implement with IBM Cloud Functions in a serverless way. During the development of that tutorial I faced the question on how to perform the database setup. Should I guide users through the user interface to create a table and insert data? Should they install a Db2 client and execute a script locally? I solved the problem in a serverless fashion. Here are the details.

Access Db2 from IBM Cloud Functions the Easy Way (Node.js)

Db2 access via IBM Cloud Functions

Recently, I have been experimenting with the IBM Watson Conversation service and Db2. With a new feature in the conversation service it is possible to perform programmatic calls from a dialog node. Why not query Db2? I implemented both a Db2 SELECT and INSERT statement wrapped into actions of IBM Cloud Functions. It is quite easy and here is what you need to know.

The conversation service supports client-side and service-side calls. This means, either the application driving the chat can be instructed to make an outside call or the conversation service itself is invoking an action. That is, IBM Cloud Function actions can be called. For my experiment I coded up two actions, one to fetch data from a Db2 database, the other to insert new data. I chose Node.js 8 runtime platform because the Db2 driver for Node.js is already part of the runtime environment. The sources for the Db2-related actions are in this Github repository.

Passing the right values to the functions shouldn't be a problem for you. Obtaining the credentials for Db2 and making them available inside the action got simplified recently. The CLI plugin for IBM Cloud Functions allows to bind a service to an action as show here:

bx wsk service bind dashDB /hloeser/ConferenceFetch --instance henrikDB2 --keyname henriksKey2

In the example I am binding credentials for the dashDB service (that is Db2 Warehouse on Cloud) to one of my actions named "ConferenceFetch". Because I have multiple service instances and possibly multiple keys (credentials) I make use of the optional parameters "instance" and "keyname". Thereafter, the Db2 configuration including username and password is available in the action metadata. In the action code I am using this syntax to obtain the "dsn" information. The dsn is used by the Db2 driver to connect to the database.

__bx_creds: {dashDB:{dsn}}

With this rough outline you should be able to get your IBM Cloud Functions connected to Db2. Those actions can then directly be called from within a dialog node of Watson Conversation. I will post details on how that works in another blog entry.

If you have feedback, suggestions, or questions about this post, please reach out to me on Twitter (@data_henrik) or LinkedIn.