Let’s write an app

The best way I find to learn a programming language/library/framework is to actually use it, not just read about it (he says writing a blog post). So to help me iron out the MANY rough edges of GoUI I’ve decided to start writing an app. A basic but (hopefully) fully functioning app. Each major change I do to GoUI will involve be either rewriting or modifying the app to use the latest and greatest features of GoUI.

I’ve decided on a basic Slack messaging app. Not because I need one (well, I could do with the RAM back if I’m honest), but because it’s features from a UI pov I think are fairly well understood. We have a list of contacts/channels, we have messages related to that contact/channel and we have an area to type. Simple, but a practical example.

The code for this is on Github . The key part I want to focus on for this small blog post is the arranging of the various VPanels and HPanels used to make the rough interface. This is just a short post on what the rough outline of the app will look from a UI point of view.

Here is the current look:

The basic outline is this:

  1. Is the toolbar. Currently just a quit button on it.
  2. Will contain the various contacts/channels that the user can select
  3. Will contain the messages related to the contact/channel selected
  4. Is where you’ll type your new message

Now for the code.

func (s *Slacker) SetupUI() error {
	// This is #1, the toolbar
	tb := widgets.NewToolBar("toolbar", &color.RGBA{0, 0, 0, 0xff})
	quitButton := widgets.NewToolbarItem("quitTBI", s.QuitSlacker)
	tb.AddToolBarItem(quitButton)
	tb.SetSize(800, 30)

	// Main VPanel of the app. Will have 2 entries in it. The first is the top level toolbar
	// secondly will be a HPanel to have contacts and messages.
	vpanel := widgets.NewVPanelWithSize("main-vpanel", 800, 600, &color.RGBA{0, 0, 0, 0xff})
	vpanel.AddWidget(tb)
	s.window.AddPanel(vpanel)

	// main Horizontal panel... first element is channels + people
	// second element is actual chat.
	mainHPanel := widgets.NewHPanel("hpanel1", &color.RGBA{0, 100, 0, 255})

	// This is number 2, the side contacts.
	// contactsVPanel goes down complete left side, 100 width, 600-30 (toolbar) in height
	contactsVPanel := widgets.NewVPanelWithSize("contactsVPanel", 100, 570, &color.RGBA{0, 0, 100, 0xff})

	// This is a VPanel will will contain both 3 and 4 (messages and typing)
	// In messagesTypingVPanel we will have 2 vpanels.
	messagesTypingVPanel := widgets.NewVPanelWithSize("messagesTypingVPanel", 700, 570, &color.RGBA{0, 50, 50, 0xff})

	// Number 3... the messages
	messagesVPanel := widgets.NewVPanelWithSize("messagesVPanel", 700, 540, &color.RGBA{10, 50, 50, 0xff})
    
	// Number 4... typing
	typingVPanel := widgets.NewVPanelWithSize("typingVPanel", 700, 30, &color.RGBA{50, 50, 50, 0xff})
    
	// Add 3 and 4 to the combined Vpanel
	messagesTypingVPanel.AddWidget(messagesVPanel)
	messagesTypingVPanel.AddWidget(typingVPanel)

	mainHPanel.AddWidget(contactsVPanel)
	mainHPanel.AddWidget(messagesTypingVPanel)

	// now add mainHPanel to VPanel.
	vpanel.AddWidget(mainHPanel)
	return nil
}

More to come soon.

GoUI… a VERY simple UI framework

Recently I’ve been wanting to develop some desktop (primarily Windows, but could also be considering MacOS or Linux) applications. Although Go isn’t the first language that springs to mind when it comes to developing an UI application the simplicity and robustness of the language still makes it my number 1 go to language. Yes, it’s my hammer… but it’s a damn nice hammer.

I’d previously used the Ebiten graphics library for Go for some games I’ve coded with my kids. But, for general “application” development (ie needing ‘widgets’ such as buttons, menus, etc etc) Ebiten isn’t really suited for that. So I’ve search about for Go UI libraries that I could use. Most out there use Cgo but I REALLY wanted my applications to be pure Go. This is purely a personal preference and I cannot give statistics/numbers on what the impact would be to use Cgo as opposed to Go. If I were to use Cgo I think the UI library of preference would definitely be Fyne. Fyne looks like a very comprehensive framework, and if you don’t mind Cgo I’d definitely recommend looking at Fyne.

There are some others about but nothing that I grabbed me. They say you should never write your own security/cryptography code and I truly believe that you shouldn’t write your own UI libraries. I’m ignoring that advice… 🙂

Hence GoUI !!

This is purely one of those “scratch your own itch” types of projects. It’s still VERY early on, but the basics of a UI library (with Ebiten doing the actual rendering) is slowly coming together. Before going through any details let me state here and now, I think programs peaked at the CLI. Secondly, I’m no UI coder… I think you’ll agree BUT in my defence I am writing this for me to consume. Bad styles be damned 😛

The basic idea of GoUI is that there are 2 broad types of graphics elements. One is a panel that can contain other panels or widgets. The other is a widget which is a basic UI element (button, textinput etc etc). The panel types we currently have a HPanel (adds things horizontally) and a VPanel (I’ll let you guess). Technically we do have other panels such as a Toolbar, but that is really just an HPanel with a little bit of custom work. Overall, if I can construct something “new” out of combining other existing panels/widgets then I’ll do it. If I need to optimise or diverge from what I already have, then I’ll make something brand new.

The widgets we have are currently:

  • ImageButton: (clicked/not-clicked images are supplied by the app).
  • TextButton: basic coloured rectangle with whatever text you want in it.
  • Checkbox: Same as TextButton but with a small box ticked or not ticked to the side.
  • EmptySpace: utter cop out by me. Used to force space between other widgets. Once I add proper padding this will probably disappear.
  • Label: box with text. Doesn’t do anything.
  • Text Input: can type in it.
  • RadioButtonGroup: This is a panel, which has a vpanel or hpanel (depending on a flag) that then has a number of checkboxes in it. The check boxes have the image (with a tick) replaced with the usual radiobutton circle. This is a good case of reusing an existing widget. If it turns out that I need to modify checkboxes enough that it wouldn’t be suitable to be repurposed as a radiobutton, then I’ll have to put in a real radio button. But for now, it’s working perfectly.

I’ve still yet to complete menus, modals windows etc etc. But as I’ve said… early days.

So, let’s try a super simple demo.

package main

import (
    "github.com/hajimehoshi/ebiten"
    "github.com/kpfaulkner/goui/pkg"
    "github.com/kpfaulkner/goui/pkg/widgets"
    log "github.com/sirupsen/logrus"
    "image/color"
)

type MyApp struct {
    window pkg.Window
}

func NewMyApp() *MyApp {
    a := MyApp{}
    a.window = pkg.NewWindow(800, 600, "test app", false, false)
    return &a
}

func (m *MyApp) SetupUI() error {
    vPanel := widgets.NewVPanel("main vpanel", &color.RGBA{0, 0, 0, 0xff})
    m.window.AddPanel(vPanel)
    button1 := widgets.NewTextButton("text button 1", "my button1", true, 0, 0, nil, nil, nil, nil)
    vPanel.AddWidget(button1)
    return nil
}

func (m *MyApp) Run() error {
    m.SetupUI()
    ebiten.SetRunnableInBackground(true)
    ebiten.SetWindowResizable(true)
    m.window.MainLoop()
    return nil
}

func main() {
    log.SetLevel(log.DebugLevel)
    app := NewMyApp()
    app.Run()
}

Let’s describe what’s going on here.

Firstly, the core of the program still calls out to Ebiten directly. Those are not wrapped/hidden yet. So you’ll see in the main and Run functions that we’ve basically making an instance of MyApp struct then we call SetupUI, set a few Ebiten flags and then call the MainLoop.

NewMyApp function calls pkg.NewWindow function. This is the MAIN window to the application. Once modals/other-windows are added this might change but for now, this creates the main UI window of a given size.

SetupUI is where you need to pay attention. The first thing we do is create a VPanel. Remember VPanels stack things vertically. We add the vPanel to the main window. In practice (currently) we should only ever add 1 panel to the main window, everything else should go into that panel. So in this case we create button1 (a new TextButton) and add that to the vPanel.

Let’s take a moment to bask in my UI skills.

*tumbleweeds….. *

Ok, lets do something more interesting. Say we want to react to when the button is pressed. The line that created the TextButton is:

button1 := widgets.NewTextButton("text button 1", "my button1", true, 0, 0, nil, nil, nil, nil)

I wont go into all the details, but the last parameter is to an event handler with the signature func(event IEvent) error. So, if we create a method with that signature and pass that in as a last parameter to NewTextButton, we’ll actually start seeing something.

func (m *MyApp) ButtonAction1(event events.IEvent) error {
    log.Debugf("My button1 action 1!!!")
    return nil
}

Then we modify the button creation to be

button1 := widgets.NewTextButton("text button 1", "my button1", true, 0, 0, nil, nil, nil, m.ButtonAction1)

Now when the button is clicked, the ButtonAction1 function will be called and we can trigger whatever functionality we like.

Easy.

Now, what about if I wanted something next to the button (and I know I’ll eventually want something below the button). What we do here is to create a HPanel and put that into the VPanel first. Then we add the button to the HPanel. If we do this, we’ll end up having code like:

func (m *MyApp) SetupUI() error {

    vPanel := widgets.NewVPanel("main vpanel", &color.RGBA{0, 0, 0, 0xff})
    m.window.AddPanel(vPanel)

    hPanel := widgets.NewHPanel("hpanel1", &color.RGBA{0, 100, 0, 255})
    vPanel.AddWidget(hPanel)

    button1 := widgets.NewTextButton("text button 1", "my button1", true, 0, 0, nil, nil, nil, m.ButtonAction1)
    hPanel.AddWidget(button1)

    return nil
}

Visually nothing would have appeared to change. We still have just 1 widget displaying.

Now, what if we add a checkbox into the same HPanel?

func (m *MyApp) SetupUI() error {

    vPanel := widgets.NewVPanel("main vpanel", &color.RGBA{0, 0, 0, 0xff})
    m.window.AddPanel(vPanel)

    hPanel := widgets.NewHPanel("hpanel1", &color.RGBA{0, 100, 0, 255})
    button1 := widgets.NewTextButton("text button 1", "my button1", true, 0, 0, nil, nil, nil, m.ButtonAction1)
    hPanel.AddWidget(button1)

    cb1 := widgets.NewCheckBox("my checkbox1", "check me please", "", "", nil)
    hPanel.AddWidget(cb1)

    vPanel.AddWidget(hPanel)
    return nil
}

So, exact same a before but just with those 2 extra lines, creating the checkbox then adding to the hPanel.

Visually we now get:

I’m not worrying about spacing here yet, but you get the idea.

Now lets add an ImageButton below the TextButton. This means we’ll be adding a second item to the vPanel ( the first item being the hPanel)

    imageButton := widgets.NewImageButton("ib1", "images/pressedbutton.png", "images/nonpressedbutton.png",nil )
    vPanel.AddWidget(imageButton)

This will produce

In this case we have 2 images (only one shown in the screen shot). One when pressed and one when not pressed (just changes the shading). You get the idea.

I’ll wrap up for here now… but so far I can produce functional (polite way of saying ‘ugly’) interfaces… but this will definitely improve!

More Logmine and patterns

As mentioned previously I’ve been developing a log mining library/tool based on the LogMine algorithm. In that blob post I was mining the mammoth log file containing 12 lines (whoa!), now for a more practical example.

Part of my day job is dealing with Jmeter. With great power comes great volumes of log files…. so time to get mining! Taking a subset of logs from a test system, I’ve got a nice set of approx 900k lines. Not huge but also large enough that if new messages of interest were to start appearing, I might not currently notice them. 200 similar log messages might be of interest, but would easily be overlooked in 900k lines.

Remembering Gologmine is still work in progress (but still definitely usable for my purposes), we can start investigating what it produces. Also remember that the whole idea of the LogMine algorithm is that individual log files are clustered together into more generic patterns. With each iteration we make each cluster more and more generic.

So, (don’t worry I wont copy all 900k log lines here), after the first level (0) of clustering, we get the patterns:

count 248 : pattern NUMBER Single Read Sampler,No * timeout, no response IPV4 xxxxxx Use NUMBER
count 549 : pattern NUMBER * - Present,Non HTTP response code: org.apache.http.conn.HttpHostConnectException,Non HTTP response message: Connect to * IPV4 failed: Connection timed out * timed IPV4 xxxxxx Use NUMBER
count 2883 : pattern NUMBER Open Connection,Websocket I/O error,WebSocket I/O error: connect timed IPV4 xxxxxx Use NUMBER
count 44966 : pattern NUMBER Single Read IPV4 xxxxxx Use NUMBER
count 45213 : pattern NUMBER Single Write Sampler - Camera Image,Sampler Error,Request data is not binary: IPV4 xxxxxx Use NUMBER
count 45214 : pattern NUMBER Single Write Sampler - Kayne IPV4 xxxxxx Use NUMBER
count 45214 : pattern NUMBER Single Write Sampler - Benny IPV4 xxxxxx Use NUMBER
count 45214 : pattern NUMBER Single Write Sampler - Vampire IPV4 xxxxxx Use NUMBER
count 45580 : pattern NUMBER Open Connection,101,Switching IPV4 xxxxxx Use NUMBER
count 48916 : pattern NUMBER * - IPV4 xxxxxx Use NUMBER
count 49731 : pattern NUMBER - IPV4 xxxxxx Use NUMBER
count 525011 : pattern IPV4 xxxxxx Use NUMBER

For this particular set of logs, most of the levels actually produce very similar output. But this is still worth noting some useful pieces of information.

Firstly, even at level 0 (most basic form of clustering) we can see that a timeout was triggered 248 times. Now, something with such as obvious word as “timeout” might already be the basis of an alert.. but what I want to show here is that in the raw logs none of the log lines are duplicated. Here we have discovered a pattern: NUMBER Single Read Sampler,No * timeout, no response IPV4 xxxxxx Use NUMBER

If we were purely looking through raw logs, these timeouts could very easily be missed. But when we are dealing with general patterns, information bubbles up easier for ops to consume.

Another item here is that we see 3 very similar log patterns referring to “Single Write Sampler”. One for Kayne, Benny and Vampire. Now, this information might be useful or maybe we’re just interested in the total logs of that type but not who it refers to. This is where moving to another level comes into play.

Lets try level 2

count 248 : pattern NUMBER Single Read Sampler,No * timeout, no response IPV4 xxxxxx Use NUMBER
count 549 : pattern NUMBER * - Present,Non HTTP response code: org.apache.http.conn.HttpHostConnectException,Non HTTP response message: Connect to * IPV4 failed: Connection timed out * timed IPV4 xxxxxx Use NUMBER
count 2883 : pattern NUMBER Open Connection,Websocket I/O error,WebSocket I/O error: connect timed IPV4 xxxxxx Use NUMBER
count 45213 : pattern NUMBER Single Write Sampler - Camera Image,Sampler Error,Request data is not binary: IPV4 xxxxxx Use NUMBER
count 48916 : pattern NUMBER * - IPV4 xxxxxx Use NUMBER
count 49731 : pattern NUMBER - IPV4 xxxxxx Use NUMBER
count 90546 : pattern NUMBER * IPV4 xxxxxx Use NUMBER
count 135642 : pattern NUMBER Single Write Sampler - * IPV4 xxxxxx Use NUMBER
count 525011 : pattern IPV4 xxxxxx Use NUMBER

By moving up a clustering level (2), the more generic the log patterns are. In this case the Kayne, Benny and Vampire are all merged into the single pattern (second last) and the count is a combination of the 3 earlier lines.

Another good example is the 4th one in the above list: count 45213 : pattern NUMBER Single Write Sampler – Camera Image,Sampler Error,Request data is not binary: IPV4 xxxxxx Use NUMBER . In this case we have 45k entries that have different details (timestamps, IPs etc) which are all unique… but here we have been able to generate a single pattern to detect this anomaly. We didn’t have to know if a string to search for, we simply had to run logmine across out logs and it automatically determined common patterns.

I’ll soon be integrating this into the monitoring/alerting system I use daily.

Code is located here

LogMine (ish) log pattern generation

Recently I attempted to implement a Go version of the LogMine algorithm ( https://www.cs.unm.edu/~mueen/Papers/LogMine.pdf ). This was a partial success… success as in I can mine logs for patterns but a partial failure in that I could understand a large chunk of the algorithm (based off the paper and other presentations/videos I could find off the net) I couldn’t understand ALL of it. So the implementation I ended up with was mostly LogMine and some "Ken filled in the blanks with what his brain could handle". Disappointing, but hey, I’ve got working code 🙂

I looked at some other projects that attempted to do the same thing (mostly Python implementations) to see if reading code would make the ideas "click" in my head, but it turns out that these implementations (to my eye) stopped even earlier than I did (ie their implementations worked for them, but wasn’t the full LogMine)

Being able to generate patterns from existing log files has mostly been an exercise in regex-fu. I wanted to avoid this and try and come up with an implementation that could work on unknown log files that didn’t required any (or at least much) human tweaking/tuning. LogMine certainly filled the bill there.

As way of a quick explanation, I’ll steal the example the LogMine authors use themselves:

2017/02/24 09:01:00 login 127.0.0.1 user=bear12
2017/02/24 09:02:00 DB Connect 127.0.0.1 user=bear12
2017/02/24 09:02:00 DB Disconnect 127.0.0.1 user=bear12
2017/02/24 09:04:00 logout 127.0.0.1 user=bear12
2017/02/24 09:05:00 login 127.0.0.1 user=bear34
2017/02/24 09:06:00 DB Connect 127.0.0.1 user=bear34
2017/02/24 09:07:00 DB Disconnect 127.0.0.1 user=bear34
2017/02/24 09:08:00 logout 127.0.0.1 user=bear34
2017/02/24 09:09:00 login 127.0.0.1 user=bear#1
2017/02/24 09:10:00 DB Connect 127.0.0.1 user=bear#1
2017/02/24 09:11:00 DB Disconnect 127.0.0.1 user=bear#1
2017/02/24 09:12:00 logout 127.0.0.1 user=bear#1

In this case you can see that if you ignore timestamps, IPs and even user names, then you can see there are basically 4 patterns of log messages that are appearing. "login", "logout", "DB Connect" and "DB Disconnect". You might be in a situation where you’re interested in these patterns.

When using the tool (or as a library) you can mine the logs and cluster the messages into groups of similar messages. Then you can keep expanding the boundaries of the clusters depending on how much detail is required).

In this specific case, we can perform the most general form of clustering with the results:

count 1 : pattern DATE TIME DB Disconnect IPV4 user = NOTSPACE
count 1 : pattern DATE TIME logout IPV4 user = NOTSPACE
count 1 : pattern DATE TIME login IPV4 user = NOTSPACE
count 1 : pattern DATE TIME DB Connect IPV4 user = NOTSPACE
count 2 : pattern DATE TIME DB Disconnect IPV4 user = WORD
count 2 : pattern DATE TIME logout IPV4 user = WORD
count 2 : pattern DATE TIME login IPV4 user = WORD
count 2 : pattern DATE TIME DB Connect IPV4 user = WORD

In this case you can see some elements of the messages have been generalised out (time, date, unique words etc)

Next, we can generalise the clustering to the next level and get:

count 3 : pattern DATE TIME login IPV4 user = NOTSPACE
count 3 : pattern DATE TIME DB Connect IPV4 user = NOTSPACE
count 3 : pattern DATE TIME DB Disconnect IPV4 user = NOTSPACE
count 3 : pattern DATE TIME logout IPV4 user = NOTSPACE

Again, we can see we have 12 entries (totalling the counts) and we can also see that in general we have 4 types of messages. login, logout, DB Connect and DB Disconnect.

If we take it one step further we start getting into the situation of very generalised messages:

count 6 : pattern DATE TIME WORD IPV4 user = NOTSPACE
count 6 : pattern DATE TIME DB WORD IPV4 user = NOTSPACE

Here we’re simply breaking it down to messages that happened to have "DB" in it and those that didn’t.

We can take it one final level, but in this situation (although producing correct results) aren’t overly useful:

count 12 : pattern DATE TIME * WORD IPV4 user = NOTSPACE

At each level, the same number of logs are being represented (12) but with varying levels of details. For this specific test log, maybe the second set of results were the most useful. At level 1 (second set of logs) we know there are 4 general sets of messages, login, logout, DB Connect and DB Disconnect.

My plan for this is to connect it to some existing logging tools I’ve got (in particular https://github.com/kpfaulkner/ddlog ) and let me detect any new patterns of logs that appear in the production logs and alert on them. I can do this without having to KNOW about every single log message format coming into the system. If a new pattern appears over X times within a certain time duration then I’ll end up getting alerted on it for me to determine if this is something I need to pay attention to or something more benign.

Link to the code is https://github.com/kpfaulkner/gologmine

Converting Wheatley (Slack bot) to Azure Functions!

I’ve first created the basis of Wheatley (my Slack bot) a few years ago during a hackathon. Now, I’ve used it at multiple companies helping me with various ops processes as well as adding in any little functionality that will help my co-workers. Normally I run it in a VM (both Windows and Linux, being that it’s Go based I can compile for most things), but have recently decided to add an option of running it as an Azure Function.

I was impressed how trivially easy this was. Kudos to both Microsoft for finally making it easy to make a Go based Azure Function, but also to Slack for allowing various alternate APIs to integrate against.

For details about making an Azure Function in Go, please see my previous post. What I’d like to highlight here is the Slack specifics I had to do to get AFs working properly. The initial hurdle was that Wheatley only used the RTM (real time communications) protocol, which is basically a fancy was of saying websockets. Now, my main aim for using Azure Functions is that I only wanted to have it running when I needed it and not to have hosted compute always running and always having connections to the various clients. Fortunately, Slack has an alternate option called Event API. Events API is basically just falling back to the good old REST protocol…. given how infrequent the messages really are in Slack (in the big scheme of things) REST works for me nicely.

Jumping across to my Slack library of choice, Slack-go also provides Event API functionality as well as the already used RTM functions. Cool… no switching libraries!

Basically the way Wheatley is designed, very little of it is Slack specific. Sure, receiving and sending messages are obviously Slack specific but those are just some very small touch points. Most of the code is integrating with other systems which has absolutely nothing to do with Slack. So, let’s look at the older RTM based code.

api := slack.New(slackKey)
rtm := api.NewRTM()
go rtm.ManageConnection()

for msg := range rtm.IncomingEvents {
  switch ev := msg.Data.(type) {
  case *slack.MessageEvent:
    originalMessage := ev.Text
    sender := ev.User
    
    // lets just echo back to the user.    
    rtm.SendMessage(rtm.NewOutgoingMessage(originalMessage, ev.Channel)
  default:
  }
}

So, we have some pre-defined slackKey (no, not going to tell you mine), we establish a new RTM connection and basically just sit in a loop getting the latest message and replying. Obviously Wheatley does a lot more, see github for the exact details.

So effectively we just need something similar without the websockets shenanigans.

There’s a bit more handshake ceremony going on, but really not much. Instead of 1 token (above called slackKey) there are 2. The one already mentioned and another called the verification token. This token is used to confirm that the messages you’re receiving are actually for this particular instance of the bot.

Fortunately our HTTP handle func is the same type we’re all used to in Go. The highlights of the function are follows:

var slackApi = slack.New(token)   // same token as RTM... 
func slackHttp( w http.ResponseWriter, r *http.Request) {

  // read the request details.
  buf := new(bytes.Buffer)
  buf.ReadFrom(r.Body)
  body := buf.String()
    
  // ug, hate the wordpress formatting, but basically we're using the 
  // Slack-go API to parse the 
  // event we've received. In this part we also confirm that the 
  // VerificationToken we received matches
  // the one we already have from the Slack portal (variable 
  // verificationToken)
  eventsAPIEvent, e := slackevents.ParseEvent(json.RawMessage(body), 
    slackevents.OptionVerifyToken( 			
    &slackevents.TokenComparator{VerificationToken: 
    verificationToken}))
  if e != nil {
        
    // verification token not matching... bugger off.
    w.WriteHeader(http.StatusUnauthorized)
    return
  }
    
  // Check that we're the bot for this acct.
  // Taken directly from the slack-go event api example :)
  if eventsAPIEvent.Type == slackevents.URLVerification {
    var r *slackevents.ChallengeResponse
    err := json.Unmarshal([]byte(body), &r)
    if err != nil {
      w.WriteHeader(http.StatusInternalServerError)
    }
    w.Header().Set("Content-Type", "text")
    w.Write([]byte(r.Challenge))
  }
    
  // Dealing with the message itself.
  if eventsAPIEvent.Type == slackevents.CallbackEvent {
    innerEvent := eventsAPIEvent.InnerEvent
    switch ev := innerEvent.Data.(type) {
    case *slackevents.MessageEvent:

      // return 200 immediately... according to https://api.slack.com
           /events-api#prepare
      // otherwise if we dont return in 3seconds the delivery is 
      // considered to have failed and we'll get another
      // message. So can return 200 immediately but then the code that 
      // processes the messages can
      // return their results later on
      w.WriteHeader(http.StatusOK)
            
      originalMessage := ev.Text
      sender := ev.User
            
      // again, we'll just echo it back.
      slackApi.PostMessage(ev.channelID, slack.MsgOptionText( 
        originalMessage, false))
    }
  }
}  
  
    
   

If you’re interested in the real Wheatley version (that’s wrapped in the Azure Function finery) then check on github.

The most awkward part is getting the bot permissions correct in the Slack Portal. So far for the basic messaging I’m needing the permissions of: users:read, app_mentions:read, channels:history, chat:write, im:history, mpim:history, mpim:read are useful. These are set in both the Event API part of the portal and the OAuth section.

After a few more days of testing this out on my private Slack group I think Slack + Wheatley + Azure Functions are ready to be unleashed on my co-workers 🙂

Remote execution of Powershell on Azure VM

From an operations point of view remotely executing commands on a machine is critical for anything beyond a few machines. In the Windows world the way I’ve usually done this is allowing remote powershell…. but I’ve recently realised (I’m slow on the uptake) that I can do this the Azure CLI. If I can do it with the Azure CLI (az) it means there is a REST API… If there is a REST API it means I can tinker.

Proceed with the tinkering!!

First thing’s first. The az command to achieve this is:

az vm run-command invoke --command-id RunPowerShellScript --name my-vm-name -g my-resourcegroup  --scripts 'echo \"hello there\" > c:\temp\ken'

Now the fun bit. To run some arbitrary bit of powershell ( which is scary enough ) the REST endpoint is :

https://management.azure.com/subscriptions/xxxx/resourceGroups/xxxxx/providers/Microsoft.Compute/virtualMachines/xxxxxx/runCommand?api-version=2018-04-01

with the usual substitution in of subscriptionID, resource group name and VM name.

You POST to the above URL with a body in the format of:

{"commandId":"RunPowerShellScript","script":['<powershell>']}

So the powershell could be the REAL commands…. format a drive, start the bitcoin miner etc etc…. OR… in my case I simply want to execute the powershell that has already been installed on the remote machine and has been verified as safe 🙂

I’m going to incorporate this remote execution into some tools I maintain for my own use (all in Go), so the entire program boils down to something pretty small. Firstly auth against a service principal in Azure, then with the generated token execute the POST. Auth with Azure is simplified with using my tiny AzureAuth project :

azureAuth := azureauth.NewAzureAuth(subscriptionID, tenantID, clientID, clientSecret)
azureAuth.RefreshToken()
template := "https://management.azure.com/subscriptions/%s/resourceGroups/%s/providers/Microsoft.Compute/virtualMachines/%s/runCommand?api-version=2018-04-01"

url := fmt.Sprintf(template, ps.subscriptionID, resourceGroup, vmName)

bodyTemplate := "{\"commandId\":\"RunPowerShellScript\",\"script\":['%s']}"
script := "c:/temp/myscript.ps1"
body := fmt.Sprintf(bodyTemplate, script)

req, _ := http.NewRequest("POST", url, strings.NewReader(body))
req.Header.Add("Authorization", "Bearer " + azureAuth.CurrentToken().AccessToken)
req.Header.Add("Content-type", "application/json")
client := http.Client{}
client.Do(req)

This is obvious with all the error checking removed etc (just to reduce the clutter here).

Now, one important thing to remember. If you want immediate (or even just vaguely quick-ish) execution of your scripts, executing Powershell via Azure REST APIs is NOT the way to achieve this. Even a simple Powershell script to write a hello world file might take 20 seconds or so.

The benefit of this (to me at least) is not enabling general remoting to the Azure VM, no fiddling with firewall rules etc etc. It’s using the Azure REST API (that I use for SOOO many other reasons), using the same type of authentication, same way to integrate into my other tools. It mightn’t fit everyones need but I think this will definitely be my remoting process going onwards (for Azure VMs)

See here for simple implementation. Don’t forget to create a service principal and assign it VM Contributor rights to the VM you’re trying to remote to!

Azure Functions and GO UPDATED!

FINALLY

Note: I forgot to mention that so far, due to a bug in the Go runtime, I can only get binaries created from Go 1.13 to work. Have filed a bug and will see how things go.

Azure has finally made it possible to write Azure Functions in any (well virtually) language you like. Yes, it’s just a redirection layer to your pre-compiled executable/script but hey, it works…. and it works with GO 🙂

Firstly, you’ll want to read about Azure Custom Handlers , also if you’re interested in Go, check out the samples . The samples include triggers for HTTP, Queues, Blobs etc. For now, I just want to focus on the HTTP triggers. They have seriously made this so easy, in particular running locally vs in an Azure Function is literally a line or two of changes.

Firstly, the image on the Azure Custom Handlers needs to be studied before we go anywhere.

Basically the Functions Host is just a redirection layer to our REAL function, which is basically a webserver. Yes, this is a hack… no question about it… BUT… it’s exactly what I’m after. We can use ANY language we want, as long as it handles HTTP request. Yes, there are overheads compared to not having this indirection layer, but really, I’m WAY more than satisfied with this approach. I’m just glad they didn’t insist we all use Docker containers for everything.

So, as long as we can run a tiny webserver we’re good. Fortunately, Go (and most languages out there these days) come with half decent HTTP servers built in.

For a simple Go example, I’m using:

package main

import (
	"fmt"
	"log"
	"net/http"
	"os"
)

func doProcessing( w http.ResponseWriter, r *http.Request) {
  fmt.Fprintf(w,"testing testing 1,2,3")
}

func main() {
	port, exists := os.LookupEnv("FUNCTIONS_HTTPWORKER_PORT")
	if !exists {
		port = "8080"
	}
	http.HandleFunc("/testfunction", doProcessing)
	log.Fatal(http.ListenAndServe(":"+port,nil))
}

It simply responds to a GET request to /testfunction with a string. Not exciting, but it will do for now. You see that the only change between local and Azure Function versions is the port. If the environment variable FUNCTIONS_HTTPWORKER_PORT exists, then it will use that as the port number, otherwise defaults to 8080 for local env.

Next there are 2 required files, host.json which basically says how the AzureFunction will run, ie what’s the executable that’s going to be the webserver. Mine is:

{
  "version": "2.0",
  "extensionBundle": {
    "id": "Microsoft.Azure.Functions.ExtensionBundle",
    "version": "[1.*, 2.0.0)"
  },
  "httpWorker": {
    "description": {
      "defaultExecutablePath": "azurefunctiongo.exe"
    }
  }
}

Where azurefunctiongo.exe is the executable generated from the above Go code.

Finally there is function.json, this is the binding about a particular function. So in my case I was interested in a HTTP trigger, so my function.json looked like:

{
  "bindings": [
    {
      "authLevel": "anonymous",
      "type": "httpTrigger",
      "direction": "in",
      "name": "req",
      "methods": ["get", "post"]
    },
    {
      "type": "http",
      "direction": "out",
      "name": "res"
    }
  ]
}

It can handle GET and POST (although my code is currently only GET). Incoming is HTTP and output is also HTTP. You might want situations where the input (ie trigger) is HTTP but the output is putting a message onto an Azure Queue for example. The Azure Customer Handler page linked above covers all of this.

Now, all of these just get uploaded to the usual wwwroot of an App Service Plan (will automate that soon) and away you go! (note, make sure you the exe and host.json are in the wwwroot directory, and the function.json is in a subdirectory of wwwroot called whatever your endpoint is called, in my case testfunction)

Now that Azure has this going, I can see SOOOOO many use cases for my projects. Thankyou thankyou thankyou Azure!!!

Github Actions in Go

After a little tinkering about with Github Actions, I decided to write one to scratch an itch I’d been having at work. Github actions react to various events , one of which is Github wiki updates (awesomely named as Gollum events). Unfortunately the various actions I tried from the market place (which take events and then ping slack with details) simply sent a message saying a wiki page was updated and never actually said WHICH page was updated. So I decided it was probably worth while to write up my own to learn how Github Actions work under the covers.

Now, most of what I learned I got off Jacob Tomlinsons great post. I won’t reproduce what he already covers (far better than I could), but just wanted to list a few points that took me a little to realise.

Firstly, it’s in the YML file (an action is configured by yaml… I wont comment) that event types are configured. So in the case of reading wiki/gollum events, the yaml file will have an "on: gollum" entry. This means that the rest of the action (which is compiled on each execution) won’t actually get run except when it’s really needed.

Secondly, if you just want to run something when a particular event is triggered you don’t actually need to read the payload (with a TONNE of info in it). This is what a lot of actions seem to do, but for my situation where I needed to know which wiki page was updated, I needed the specifics. For the specifics you’ll need to read an environment variable (see big list) called GITHUB_EVENT_PATH. This is a path to a file that you then read, and this file has all the specifics of the event triggered. Although some might scoff at the relatively low tech approach to this, I really like it. Read filename from environment variable, then read the file. Easy peasy.

The file is simply JSON (see the events link earlier) and you can deserialise into a class/struct/whatever-suits-your-language easily enough. From there, grab the info you want and pass that to Slack. In the case of Go, the wonderfully useful Json-to-Go site makes this trivial.

Now that I’ve got to grips with writing actions in Go, I’m certain I’ll be writing more. There are many cases where I can picture them being VERY useful in my day-to-day work.

The particular action I wrote up is precious

When “good enough” is really good enough

We all (sweeping generalisation) get analysis paralysis at times. Or at least I think we (devs) all do. I know I certainly get it way too often for me to be comfortable. I’ve always been around REALLY good developers…. people that can out-code me in a minute, and that is REALLY intimidating. They come up with solutions that appear (and possibly are) complicated and well thought out. They’re breaking systems up into nice controller, services, data layers etc etc with what seems like endless abstractions. They’re good… they’re the top devs in the places… what am I not "getting" to see that all of this work is absolutely needed. Yes, they might know that the design they’ve come up with will be perfect for when addition 1,2 and 3 come along.

But what if the additions don’t come? Or they do, but (cue The Imperial March) "the business" have skewed the requirements enough that they appear very different to what was initially talked about.

I’ve ALWAYS preferred simple code rather than clever code, even if it makes me look foolish in front of other developers (probably not a great career move… but hey 23 years in and I’m still going ok). In C# I’d prefer to start with basic loops and THEN figure some uber-l33t LINQ to come up with the same results, although more often than not I’d just stick with the for/while loops. To me, they’re definitely more readable and understandable than the 3 line LINQ expression. But I certainly know other devs that instantly go to LINQ and know it inside and out (like I said, they’re far better than I am).

But… in saying all this, I came across a tweet recently by Alex Gurney ( https://twitter.com/ajtgurney/status/1230580904944439301 ). Unsure how Alexs tweet got on my timeline, but the thought about a "2 hour hack" really started to speak to me. Yes, if we KNOW …. and I mean REALLY know that what we’re coding up is just the basis for something more complex in the next sprint/iteration then yes, obviously put more thought into the design. But if a set of requirements are "do XYZ, take this input… output looks like this…. etc " then I’m starting to wonder is the 2 hour hack really a bad thing?

Of course there are many situations where this would absolutely be irresponsible to do, but equally I can picture MANY classes/services that I’ve seen (or written) where banging something out would have equally done just as well.

Then we need to address the infamous 2 x four letter words… "tech" and "debt". If we just "hack" absolutely everything then we’d probably be sitting on a house of cards, and as soon as something major needs to get worked on we’ll be stuck in a bad position. I think of "hacks" in 2 ways. The first is the less pleasant version…. do whatever it takes to get functional code (functional for the specs we have NOW). This is usually falls under the Perl category of coding (as I call it)…. made sense when you wrote it… come back a week later and it’s all WTF?!?!

The second option is where the solution is the simplest possible (within a small timeboxed period) that can be designed and coded up and isn’t refactored to be the smallest, tightest most elegant code you can create. To me the key thing is SIMPLE. I’m not talking lines of code (since I personally find fewer lines of code != simple), but I’m talking concepts… something my brain can handle.

Stick to simple functions, simple loops, don’t go and start making a bunch of abstractions/interfaces/wrappers that you think you might need in the future. The number of times I’ve seen DB or storage wrappers to abstract away the platform specifics and only end up with a SINGLE concrete implementation. The problem is, I sometimes feel I’m in the minority here. What I find simple and readable others complain isn’t. They’d prefer to see a 1 liner calling map/fold/whatever but I know I’ve certainly experienced issues debugging a map call where 1 entry in a collection has caused the mapping function to blow up. Personally I find a for-loop easier to debug, but maybe that’s just me.

Just as I’m about to post this… I see Nick Craver come up with a perfect summary of how I feel, "Don’t let perfection stand in the way of a net win." https://twitter.com/Nick_Craver/status/1231591808007798784 He mightn’t be talking about 2 hour hack, but I still like the tweet 🙂

Maybe the readability thing is why I find Go such an attractive and productive language. Yes, you can argue lack of map/fold etc and the constant err checking causes more boiler plate, but seriously that boiler plate takes a few mins to write out and then is just left there to run.

UDP VPN Load testing fun :)

Recently I was required to load test a new VPN system being installed. The VPN in question was being used to handle VOIP calls, so we needed crystal clear connections without any lag, stuttering or badness ™.

The initial set of requirements was that we’d need to be able to handle 65000 bps (UDP) per VPN connection with a minimum of 1000 concurrent VPN connections. There was no tool that easily fitted the bill for this (at least none that we could find), so here was an opportunity to roll our own. Overall this isn’t a lengthy process but definitely has a few gotchas.

Firstly, how to create 1000 VPN connections. Ideas ranged from spinning up 1000 VMs (which would auto load the VPN client), through to Docker and then finally seeing if we can really fire up that number of connections on a single machine. As long as the client machine could be able to create that number VPN tunnels I was sure the actual pumping of data shouldn’t be an issue. Now, for testing a VPN device it shouldn’t matter about the OS on the 2 machines that are communicating via it…. right? Although these days I’m more of a Windows person than a Linux one, I thought there was more chance of multiple VPN clients working on Linux.

Though endless research, experimentation and intellectual debate with others (ok, so I read StackOverflow…), it turns out setting up the VPN requires 2 steps. The first is to actually perform the VPN connection (in my case using OpenConnect). The key after that is setting up the routing properly. I’m no networking god, so this stumped me for ages. But after the OpenConnect stage, the key commands are:

sudo ip route add $destserverip dev $interface src $ipaddr table $table 

sudo ip route add default via $ipaddr table $table 

sudo ip rule add from $ipaddr table $table

Where $destserverip is "server" listening IP address, $ipaddr is the IP for a specific VPN interface and $table is a brand new table name (table1, table2, table3 etc) specific for $ipaddr.

So the above 3 lines are executed 1000 times (or however many VPN connections you make).

Once we have the VPN connections setup, we have to setup the "server" (listening service) on a box/VM and the client application that will send traffic across all VPN connections.

The "load test application" (glorified UDP pump) is written in Go (of course) and covers both server and client functionality. It’s a very simply program, from a server point of view. In it’s most simplistic form it really is just a single goroutine that accepts a UDP packet, tallies how many it’s received…. and well, that’s it. Not really news worthy. In a slightly more complex form (since this was written to test VOIP, and usually conversations are bi-directional), it spins up a separate goroutine for each incoming IP address (denoted by the VPN tunnel) and responds to the originator by echoing back exactly the same packet as it just received.

Not particularly riveting stuff, but it does the job. Accepts all the incoming traffic (from 1000+ clients and informs us of the incoming network volume).

The client side of the code is far more interesting. There are a number of factors that we want to tweak while performing these tests, bits per second, number of bits/bytes/kb per UDP packet, number of "clients" we want to simulate, number of VPN connections we want to use while simulating those clients (most of the time #clients and #VPN connections will be the same), duration of tests etc etc.

Basically the client spins up "num-clients" goroutines, each using (in most scenarios) a unique network (VPN) interface. From there each goroutine simply loops based on the input parameters (duration of test, # packets etc etc). For the tests performed, the client machine (basic 2 core VM) was able to generate enough voice quality VPN traffic to simulate 1100 user and still be mostly idle.

Usage:

​ on server:

​ ./udptool_linux_amd64 -server

​ on client:

​ ./udptool_linux_amd64 -minclientaddr "192.168.0.0.1" -noclientaddr 1000 -host "1.2.3.4:5001" -noclients 1000 -seconds 6000 -verbose -bps 65000

Broadly, the explanation of the above:

-minclientaddr is the “lowest” IP address generated by OpenConnect.

-noclientaddr specifies how many IP addresses (VPN tunnels) the client app should attempt to connect to. When determining which IP each client should use, it is basically a loop starting at “minclientaddr” and incrementing from there.

-host is the server IP:Port. By default when running udptool as server, the port is 5001.

-noclients is the number of clients we should try and simulate. Most of the time this would be the same as -noclientaddr but sometimes (if you want to double up clients on a single client IP) they could differ

– seconds How many seconds the test should run for.

– verbose Gives updates on when requests are sent and more importantly how often the client sleeps between sending packets. Given udptool is used to send X bytes per second, once it’s sent the data it sleeps until it next needs to run. How long it sleeps is displayed on the console. Most of the time it will sleep for 980+ ms (ie it spends most of it’s time sleeping). If it is ever observed that it’s NOT sleeping most of the time, it means the machine is overworked/underspecced.

-bps 65000 says to send 65000 BITS per second per client.

If anyone is interested in trying it out, please grab it for github.