Its_All.Lost
Because hope just breeds everlasting misery…

Node-Red, UIBuilder and Buttons!!

Node-Red, UIBuilder and Buttons!!

If you’ve been here before then you know I use Node-Red, if not and you don’t know what Node-Red is, you should check it out. To get a frontend UI out of Node-Red there has always been the Dashboard node for us to use and a set of Dashboard nodes to go with it. I have never been a huge fan of the Dashboard, but it was what we had and it did the job. The Dashboard nodes allowed you place UI nodes and connect them to your data and it would do the rest, no HTML code required on your behalf. But that was one of the limitations, was that you did not have control over the code. Enter UIBuilder.

The UIBuilder node is amazing. It is a Node-Red interface builder that gives you complete control, over everything. The downside (if it is one) is that you must know how to code HTML and JavaScript. UIBuilder lets you use any frontend framework you want to build your interface to Node-Red. JQuery, VueJS, MoonJS, Angular, etc.. So far I have dropped in a Bootstrap-Vue admin template and I have been going to work on that. It has taken me a minute to get used to how UIB all works, and I am not really all that great with JavaScript but I seem to be figuring things out. Hey, I have shit working.

The UIBuilder Wiki and the Github for the author have a ton of information and examples of how to get a few basic things done. Just expect most of them to be done with VueJS. It appears (at least to me, I could be wrong) that UIBuilder was originally VueJS heavy/dependent and as it matures it is more open to other frameworks. As of v3.1.3 it no longer installs Bootstrap-Vue and VueJS by default, as it had done it all prior versions. So just be aware that you can use any framework you want, but all of the examples (so far) are all written in VueJS. There is also the Node-Red forums if you get stuck, the author also happens to frequent the group.

View Post

Retrieving my IP address Remotely

Retrieving my IP address Remotely

I posted a few years ago about using Twilio and Node-Red to send/receive SMS messages. Using that flow you gain the ability to text your server and have it read back values. Any values you want, as long as you have them stored as a global some where along the way. If you haven’t already done it toss $20 on an account at Twilio for a year and get to it, its so worth it! Here I am going to add a new reading to my setup.

There are occasions when I need to access my home network when I am, well, not home. Everyone screams use a VPN! Yeah, of course, I have one, and I use it. But my problem is I have a dynamic WAN IP and I am not paying for a static IP. It doesn’t change very often but it always seems to when I need to use it the most. Step in Node-Red and Twilio. With both of those at my disposal I can now send a text message to my home server and have it respond with my WAN IP address. I can then modify my VPN settings on my phone or whatever if needed. Fucking fantastic! I had been using Twilio/NR for sensor readings and alerts but I had never thought about using it to report my IP. Theres a few ways to do this.

Lets grab our IP (the flow)

(This flow grabs the WAN IP via an exec node)

First we use an inject node and set it to fire off once on start, this grabs the IP first thing and makes sure we have some data to work with. Then we move off to an execute node. We are gonna execute some command line code and echo it back, then we’ll save it in a file and as a global. This goes in the exec node:

wget -qO- http://bot.whatismyipaddress.com/ ; echo

This goes out to whatismyipaddress.com and echoes back our IP, simple and very fast. There are a few other sites that can be used also. Ifconfig.io and ipinfo.io both are alternative sites (you just have to change the truncation from 5 to 6 characters to drop the return sign in the following function node).

Alternatively, and preferably we can use an HTTP node instead of an exec node. What? Yeah I just found this out.

(This flow grabs the IP via an HTTP node)

That will get you your WAN IP quickly via an HTTP node formats it then saves it to a file. We have to comment out the substring trimming command if we use the HTTP node over the exec node. Ok no problem. Don’t forget.

After we grab the IP we need to format it and save it as a global for Twilio (and Alexa!).

This will trim the returned output from the website to 6 characters, dropping off anything extra. It also saves our trimmed value as a global. Then we slap on a debug node and a file node to save the IP to a file for later use. You don’t have to save the IP to a file, I just like to in the even the server is restarted/power loss.

Thats it! Once the IP has been picked up and set as a global you are good to go (as long as you followed the other flow for Twilio I posted previously).

 

It figures…

After typing this up and saving the draft I decided to update NPM/nodejs/Node-Red. Upon updating I came across a Node-Red node, node-red-contrib-ip.

A simple install via Palette Manager (which doesn’t work for me) or manually installed via npm install node-red-contrib-ip and you’re good. The node works simply, nothing to configure. Add an inject node to trigger it and it spits out your machines IPv4 and IPv6 address and well as your WAN IP address. Just have to format the data the way you want and boom. I did notice it is slow, painfully slow. It doesn’t need to be lightning quick but it is noticeable compared to the wget option above. Choose your weapon, either method works. Heres the flow for using the IP node.

(This flow grabs the WAN IP via the node red IP node)

After the IP node put a function node and fill it with this code

This drops all the other node info and just gives us the WAN IP. Easy as pie.

 

Lets get notified

So far we check our IP every 12hrs and write it to a file and a global for use. Thats great, we can take it one step further. Want to get a notification when the IP does change? Lets go.

(This flow grabs the IP via an HTTP node and checks if it has been updated, then sends a notification if it has)

  • Inject and repeat every 24hrs at a set time
  • Read the stored IP address by reading the file
  • Get the current IP address and set it as a global
  • Cross check to see if the new IP matches the old IP and save it to a file if it is newer.
  • Format the message payload and send a notification (however you like, pushbullet, email, Twilio etc)

 

Check out this site, its where I got the notifications from https://steve.zazeski.com/get-a-notification-when-your-wan-ip-changes/.

Custom Alexa Node-Red Skill (revisited)

alexa-nodered

Finally! I am surprised it didn’t even take as long as it usually does for me to figure shit out. Took a few weeks. As usual, I did not come up with the solution on my own but found it out on the web and slapped it all together. I wanted a custom Alexa Node-Red skill, to be able to take a command given to Alexa and have it read back data from one of my sensors. Things like temperature sensors, water level, etc. I wanted to be able to ask Alexa what the values are. What I got: exactly what I wanted. It all works. There is two parts to this: the Node-Red flow and the Alexa skill.

Alexa Node-Red flow
Alexa Node-Red flow

First off, to get any of this working you must have your Node-Red server accessible from the outside world. That means port-forwarding, DNS, domains, SSL, all that. It’s fun getting it all working. Not. Just like my previous post, I happened to have it already setup. Once your Node-Red install is available from the web you are good to go. Now you don’t need the entire NR setup opened up either. I just allowed a few NR served pages to be available. Not the entire NR itself.

Update: I made a new post about Node-Red behind a reverse-proxy/SSL

Let’s Begin

It starts with a regular HTTP node to a switch node. That switch node splits up Alexa’s requests to NR; LaunchRequest, IntentRequest, SessionEndedRequest. LaunchRequest gets invoked when the skill starts. You could have Alexa say “Hello what do you want?” for example. IntentRequest is the goods. Then theres SessionEndedRequest, I’m assuming this gets called at the end. Haven’t toyed with it. Then you pass those requests off to do other stuff, like the DoCommand where it grabs your intent. Then a function node to extract the commands, which gets passed off to another switch node to split up the possible commands you can give Alexa. Give her as many commands as you want, then there is a “device doesn’t exist” at the bottom. This is used if she didn’t hear you right or the device doesn’t exist. All that data gets passed to a template to format what Alexa will say and sticks the data in JSON. Bam! That wasn’t so hard right?

Here’s the whole flow (all standard nodes used):

That’s the Node-Red half. You are not done yet. On to the Alexa skills half. This part is easy don’t worry. Login to your Amazon Dashboard and click Alexa. Choose “Get Started” with the Alexa Skills Kit, click add a new skill. Under Skill Information give it a name and choose the invocation word, what you will say to Alexa to start your skill. I chose “Node Red”, so I have to say “Alexa, ask Node Red….”. These can be changed at any time it seems. You won’t be publishing this skill, it stays beta for only you to use. For the Global Fields section, no you will not be using an audio player. Well, maybe you will but I didn’t, and it will probably change things for you.

Note about the flow: The NR flow works (for me) just fine however I noticed it throws an error in the debug tab whenever a command is called. If it is an unrecognized command response it doesn’t throw the error though. It complains about headers already being sent. I will update the flow if I find a fix for it.

Interaction Model

Intent Schema

Intent Schema

This is the part of the Alexa skill where you tell it what to do. It is pretty straight forward. Just copy this to your “Intent Schema”. There are no custom slot types and no values to enter.

Sample Utterances

Sample Utterances

This is where you list the invocation phrases that will activate Alexa. Normally (and in other online tutorials for Alexa skills) this is where you add a ton of different phrases. But we are not. Node-Red is going to handle that side for us. This box just gets one line of text.

Configuration

Global Fields/Endpoint

For a service endpoint you are going to pick “HTTPS”. In a lot of other tutorials you will usually choose AWS Lambda but we are doing all of our own heavy lifting with NR. We don’t need no stinking Lambda. Choose your closest location and enter the URL that your Node-Red is accessible from (via the web remember). Say no to account linking and you can also leave Permissions alone.

SSL Certificate

Certificate for Endpoint

Choose the option that bests describes you. Most likely it will be the first option. For me I am using a subdomain that is already SSL’d with Let’s Encrypt so I choose the second option.

Test

Basically just leave the toggle flipped to enable the skill for you to use. you don’t need to do anything else on this page.

Publishing Information

Nothing to do here, you won’t be publishing this skill. Why? Because it requires too much setup on the users behalf. I don’t think Amazon would approve a half functioning skill that requires advanced user setup to get working. You could always try. Good luck.

Privacy & Compliance

Three no’s and one box to check, I mean as long as it all applies to you right 😉

Done

That should be it. With Node-Red available to the web and the flow implemented the new Alexa skill you just made you should be good to go. I hope you found this useful, I sure wish I had found a blog post like this. Now go test it out with your Amazon Echo/Dot!

At the time of this writing a beta product appeared in the Amazon Dashboard for a the “Skill Builder”, looks to be a new UI for building Alexa skills. If this gets implemented for everyone in the future things may be different than they are described in this blog post.

 

Revisited

Originally posted April 26, 2017 @ 17:56

I decided to come back to this post. I was adding and modifying some things on my flow and I was using this post as a reference and decided it was not cutting the mustard, it felt unfinished. So here we are.

The above section contains the main flow you will need and it steps you through the Amazon Developer side of things that needs to be setup. Once that is all finished you should have a working flow with Alexa responding accordingly. What I feel I left out was how to configure the flow itself. Many of you may already have figured it out or can see whats going on and thats cool. Heres for the ones that need the help (myself included).

The flow

Voice requests
Request (your questions)
Question function
Question function (global request)
Alexas response
Alexas verbal response

HTTP node

The first node in the flow is the HTTP request. This is the page you will use to point your reverse proxy to. This is the page that Node-Red will server, that Amazon/Alexa will look for. This page needs to be accessible from outside your network.

Request Type

This contains the types of requests we can send to Alexa. We are only worried about IntentRequest  right now. Play with the other later.

Intents & Extract Commands

There is no need to modify these nodes. They contain the code needed to send requests.

Request

Add your verbal question here. Add it how Alexa hears it. This may take a little trial and error depending on how you talk and how she hears you. Sometimes you can simply put exactly what you are going to say in there, like “garage temperature” works fine for me but if I ask Alexa what the outside temperature is she doesn’t know and I get the unrecognized question response. But this is ok, pay attention to the output of the unrecognized responses. It will spit out what Alexa heard and how she heard it. So when I ask what the outside temperature is she hears “i outside temperature” for some unknown reason. So if I modify my request to be the same, and it works.

Function node

This function is very simple. It just grabs the global variable name you put in it. So when I ask for the “garage temperature” it looks for the global I have specified, in this case contex.global.garagetemperature. Add whatever global you want. Just make sure it is initialized first (has some data to report).

Formatting Alexas response

This template node contains the information for Alexas verbal reply. Once the basic structure is there all you do is edit the “text” to the response you want to receive, in plain english. She will respond with “exactly” what you type in there.

JSON & HTTP Response

Move along, noting to see here. The data gets formatted as JSON and the HTTP node completes the whole flow.

Done.

 

There you go. I feel better now. I at least explained WTF is going on here rather than dumping the flow on you and walking away. Sorry about that. Now you can run off and play with Alexa and Node-Red to your hearts content. The only thing I have noticed with the whole flow is that after Alexa responds Node-Red throws an error in the debug log. It all works 100% and works well but it always sends this error, and I haven’t figured it out. But I also haven’t been looking for it. Just a little FYI.

"Error: Can't set headers after they are sent."

 

This is where I found the goodness, buried deep in comments on a (awesome) blog.

Reverse Proxy with Node Red

Node-Red SSL

To use Node-Red out in the free world it needs a chaperone, that happens to be reverse proxy at the moment. If you are starting off from the beginning I would probably recommend following this guide. That’s assuming you have an Ubuntu server and are running Node-Red from there and you don’t mind running Nginx. What if you already have a server that may or may not be Ubuntu. Maybe you don’t like Nginx or maybe you already have Apache2 running. Or what if you are running Node-Red on a Raspberry Pi? Then that guide doesn’t work necessarily.

I searched all over again (briefly) on setting up Node-Red with SSL and or reverse proxy and it seems everyone has done it but no one is willing to share steps to do it? Is it that easy that it doesn’t need to be said? I wish I had found a write up to point the way, as it is simple but not necessarily easy. Especially for the average or new RPi tinkerer.

This is how I got it working. There may be other ways, and better ways but this is what I found worked for me. YMMV.

 

Prerequisites

  • A TLD domain pointed at your server.
  • A server connected to your network running Apache2 (could be a Pi!)
  • Your server open to the world (port forwarding required)
  • Legit non-self-signed SSL certs. Let’s Encrypt is free.
  • mod_proxy installed and enabled on the Apache server, here’s a good guide (only step #1 is needed)
  • Node-Red, either on the same server or another (like a RPi) doesn’t matter (same network)
  • Don’t forget to enable Node-Red admin password protection too.

 

I am going to assume that you already own the domain and have it setup and working so that you can go to https://yourdomain.com and it pops up. HTTPS, with a green padlock. If not, you need to buy the domain and point the DNS to your servers IP and create SSL certs, port forward yada yada. I host a server at home, makes it easy. For example, example and example. Oh and here are some Let’s Encrypt examples for Nginx and Let’s Encrypt on Apache.

You will need to edit your sites-available conf files and add the stuff for reverse proxy. You need more than just Node-Reds base URL to have a reverse proxy. I found it was best/easiest (shit maybe its required I don’t know, I’m just dangerous) to add a line for each webpage I was going to use specifically. So far thats Twilio voice, sms and Amazon Alexa.

 

Here is my Ubuntu Apache2 site-available conf for reverse proxy:

 

I’ve got a reverse proxy for each call I am going to make within Node-Red when using Twilio. These need to be accessible from the free world (to Twilio or Amazon etc). If you need more pages available to the world toss em in.

For example https://mydomain.com/twiliosms is what would be available to the world now.

Restart apache and you should be good to go.

 

Securing Node-Red
https://hoast.dk/wordpress/2015/10/24/securing-node-red/
https://www.hardill.me.uk/wordpress/2015/05/11/securing-node-red/
https://nodered.org/docs/security
https://nodered.org/docs/configuration

Node-Red-Admin (to hash a password)
https://nodered.org/docs/node-red-admin

Adding SSL to Node-Red
http://industrialinternet.co.uk/node-red/adding-https-ssl-to-node-red/

Let’s Encrypt SSL Certificates
https://daplie.com/articles/lets-encrypt-on-raspberry-pi/
https://www.linode.com/docs/security/ssl/install-lets-encrypt-to-create-ssl-certificates

https://gist.github.com/jhaury/10508bfef231bc09c4eae4723645b4f0

Node-Red IVR with Twilio

Twilio

A while back I posted about using Twilio and Node-Red to send/receive SMS messages. I was fiddling around with some older project stuff (uncompleted) earlier today and bounced into the IVR setup I had. Wondering if it was working I called it, and no it wasn’t. So I had to fix it. It took me a minute to come across all the related pages needed to get it working again so I decided to compile them here.

What is an IVR?

Well if you Google IVR you get: Interactive voice response (IVR) is a technology that allows a computer to interact with humans through the use of voice and DTMF tones input via keypad.

And that basically sums it up. It allows you the person, to call a computer and have the computer know what you want when pressing 1, or 5. Similar to a phone switching system but they are usually used to report some kind of information instead of route calls. IVRs basically cut out someones job once upon a time.

A better example is of a field technician calling an IVR system to read back what signals were received on the clients security system he just tested.

What we are doing here?

Were are going to get a basic IVR running on our Node-Red server to interact with and read back data or to activate/deactivate things over MQTT. If you would rather (and/or) use SMS to achieve kinda the same thing read here.

Prerequisites

 

The Flow!

the flow

Here is the glorious flow!

You will need to create a global and have its value set so the IVR can read it back (in this example).

Drop a line in the comments if you have a question.

 

What helped:

https://flows.nodered.org/node/node-red-node-twilio
old outdated flow https://flows.nodered.org/flow/637b5f6128a8d423503f
https://groups.google.com/d/topic/node-red/yAEx5k2UHJw/discussion


[{"id":"4868a284.66517c","type":"http response","z":"feaafa48.a34f88","name":"Send TwiML","x":870,"y":200,"wires":[]},{"id":"1444c0f8.9e1bff","type":"http in","z":"feaafa48.a34f88","name":"Incomming Twilio Call","url":"/twiliovoice","method":"post","upload":false,"swaggerDoc":"","x":316,"y":202,"wires":[["8cebf57a.769bc8","a77ee63.fbb7618"]]},{"id":"8cebf57a.769bc8","type":"function","z":"feaafa48.a34f88","name":"Process Call","func":"var twiml_response = \"<?xml version=\\\"1.0\\\" encoding=\\\"UTF-8\\\" standalone=\\\"yes\\\"?> \\\n <Response> \\\n <Gather action=\\\"https://yourserver.com/twilioivr\\\" numDigits=\\\"1\\\"> \\\n <Say voice=\\\"woman\\\">Welcome to Node-Red. For power usage press 1. To ring the bell press 2. For anything else press 3</Say> \\\n </Gather> \\\n <Say voice=\\\"woman\\\">Sorry, I didn't get your response.</Say> \\\n </Response>\";\nmsg.payload = twiml_response;\nreturn msg;","outputs":1,"noerr":0,"x":630,"y":200,"wires":[["4868a284.66517c","daf3c74e.9d90e8"]]},{"id":"2e446272.06064e","type":"http in","z":"feaafa48.a34f88","name":"IVR Data Call","url":"/twilioivr","method":"post","upload":false,"swaggerDoc":"","x":290,"y":280,"wires":[["f769d46d.3e7fc8","a306c4e8.68fd58"]]},{"id":"daf3c74e.9d90e8","type":"debug","z":"feaafa48.a34f88","name":"","active":true,"complete":false,"x":870,"y":280,"wires":[]},{"id":"f769d46d.3e7fc8","type":"debug","z":"feaafa48.a34f88","name":"","active":true,"complete":false,"x":630,"y":320,"wires":[]},{"id":"a77ee63.fbb7618","type":"debug","z":"feaafa48.a34f88","name":"","active":false,"console":false,"complete":"payload","x":630,"y":160,"wires":[]},{"id":"a306c4e8.68fd58","type":"function","z":"feaafa48.a34f88","name":"handleGather","func":"// Use 2 outputs, 2nd is for mqtt topic/payload combination\n// Could be useful to post other caller info to a topic\nvar mqttOut = null;\nvar responseMsg = \"\";\n\nvar name = \"wholehouse\";\nvar reading = context.global[name];\nvar digit = msg.payload.Digits;\nif( digit == \"1\" ) {\n var twiml_response = \"<?xml version=\\\"1.0\\\" encoding=\\\"UTF-8\\\" standalone=\\\"yes\\\"?> \\\n <Response> \\\n <Say voice=\\\"woman\\\">Power usage is \" + reading + \"</Say> \\\n </Response>\";\n msg.payload = twiml_response;\n} else if( digit == \"2\" ) {\n\tmqttOut = {topic: \"bellduino/ring\", payload: \"ringmybell\" };\n var twiml_response = \"<?xml version=\\\"1.0\\\" encoding=\\\"UTF-8\\\" standalone=\\\"yes\\\"?> \\\n <Response> \\\n <Say voice=\\\"woman\\\">Turing on the bell</Say> \\\n </Response>\";\n msg.payload = twiml_response;\n} else {\n\tresponseMsg = \"You pressed \" + digit;\n}\n\nreturn [msg, mqttOut];","outputs":"2","noerr":0,"x":640,"y":280,"wires":[["4868a284.66517c","daf3c74e.9d90e8"],["a023a81c.a3e318","442a4b04.0c3c64"]]},{"id":"413a0cf4.24b0e4","type":"globalGetSet","z":"feaafa48.a34f88","name":"Text Global","topic":"","context":"msg","variable":"payload","outContext":"global","outVar":"wholehouse","x":490,"y":380,"wires":[[]]},{"id":"390ea068.1c8f8","type":"inject","z":"feaafa48.a34f88","name":"","topic":"","payload":"42.1","payloadType":"num","repeat":"","crontab":"","once":true,"x":290,"y":380,"wires":[["413a0cf4.24b0e4"]]},{"id":"a023a81c.a3e318","type":"mqtt out","z":"feaafa48.a34f88","name":"","topic":"","qos":"","retain":"","broker":"bdbaaa3f.072c48","x":850,"y":400,"wires":[]},{"id":"442a4b04.0c3c64","type":"debug","z":"feaafa48.a34f88","name":"","active":true,"console":"false","complete":"true","x":850,"y":360,"wires":[]},{"id":"bdbaaa3f.072c48","type":"mqtt-broker","z":"","broker":"localhost","port":"1883","clientid":"","usetls":false,"compatmode":true,"keepalive":"60","cleansession":true,"willTopic":"","willQos":"0","willPayload":"","birthTopic":"","birthQos":"0","birthPayload":""}]

view raw

node-red twilio IVR

hosted with ❤ by GitHub

 

SEND IT.