Ben Godfrey
Sep 18, 2014

September London PhoneGap Meetup

This week we hosted the London PhoneGap Meetup at the Wonga offices in London.

PhoneGap allows us to build mobile apps for many different platforms using HTML. Lots of our developers already know HTML and native development has a steep learning curve, so this is a big win for us.

The London PhoneGap meetup is a regular get together for developers using PhoneGap. It’s a place for us to share ideas and techniques and to discuss ways to make building mobile apps even easier.

This month we had 3 talks:

On September 30th, we’ll be hosting the next AngularJS London meetup. I’ll be talking in detail about acceptance testing with Cucumber.js and Protractor.

Keep up to date with events at Wonga by following @wongatech. See you at an event soon!

#phonegap #appium #firefoxos #cucumber.js #protractor
Mar 14, 2014

Generate JSON Index Of Images With awk and identify

I maintain a manifest file for a Trigger.io mobile app which includes a set of images indexed by size. I’ve taken to generating the JSON with awk and Imagemagick’s identify command:

identify returns a wealth of information about an image file:

$ identify Default~iphone.png
Default~iphone.png[6] PNG 320x480 320x480+0+0 8-bit sRGB 11.5KB 0.000u 0:00.000

I want to transform that into some JSON. awk is designed for exactly this kind of text transformation problem.

$ identify *.png | awk '{gsub("\.png.*$", ".png", $1); printf "\"%s\": \"%s\",\n", $3, $1}' | sort
"1024x748": "Default-Portrait@2x~ipad.png",
"1024x768": "Default-Landscape~ipad.png",
"2048x1496": "Default-Landscape@2x~ipad.png",
"320x480": "Default~iphone.png",
"640x1136": "Default-568h@2x~iphone.png",
"640x960": "Default@2x~iphone.png",
"768x1004": "Default-Portrait~ipad.png",

Copy and paste into manifest, remove trailing comma, make cup of tea.

EDIT: Achieve the same effect with the -format parameter for identify (-format docs here):

identify -format '"%wx%h": "%f",\n' *.png
#awk #imagemagick #unix
Feb 15, 2014

Type-check Your JavaScript With TypeScript

TypeScript is a superset of JavaScript with optional static typing. The DefinitelyTyped project contains type definitions for major JavaScript libraries contributed by the community. You can use these definitions and TypeScript’s compiler to check your code for errors, just by adding references.

We have a large AngularJS project. I’d like to type-check all our code and maybe find a few bugs.

Install TypeScript

Clone DefinitelyTyped to references

The DefinitelyTyped project provides type definitions for AngularJS, Jasmine and about 100 other common JavaScript libraries.

To keep it simple, I’m just going to clone the whole lot into my project, where I can reference anything I need.

Rename All .js Files To .ts

By renaming all our files, we let tsc and other tools know the code should be treated as TypeScript.

Let’s give our unix skills a workout and do this with a find one-liner:

Create References Includes For The Project

TypeScript code needs to be annotated with comments to link to the type definitions we downloaded from DefinitelyTyped. E.g. for a Jasmine spec which tests an Angular controller, we might need the following block at the top of the file:

You’ll need a reference for each library you’re using. To simplify things, let’s create a couple of reference files which define the shared references for our module definitions and test specs and include those in our code.

Note that test.ts references module.ts.

Add References To All .ts Files

Now, let’s add the references to every TypeScript file with a couple more find one-liners (requires GNU sed, if you’re on OS X, install with brew install gnu-sed --default-names).

Phew!

The path attribute needs to be right. Luckily, all of the code in our project is 5 levels deep in the directory structure. If your code doesn’t have such a regular structure, you’ll need to do something more clever.

Compile Everything With tsc

Finally, let’s run everything through the TypeScript compiler and see the issues we have to resolve:

Adding Definitions For Missing Modules

Adding type definitions for your own modules or 3rd-party modules missing from DefinitelyTyped is straightforward. A few examples:

#typescript #definitelytyped #javascript
Feb 6, 2014

“Bubble” Lines Up And Down Like Sublime Text In Vim

NB: You can see this hack in my .vim repo on GitHub.

Sublime Text allows you to use ^⌘↑ and ^⌘↓ to “bubble” a selected block of text up and down in a file.

For a single line, we could just use the dd, j, p and P motion and editing commands to move the current line around:

However, this doesn’t work for multiline selections and it overwrites the default register, wiping anything you’ve previously deleted to there.

A better solution is to use the :move command. We can safely use it in both normal and visual mode because it works with multiline blocks:

We pass locations relative to the selection to :move, '<-2 is 2 lines before the first line of the selection, '>+1 is one line after the last line.

In the visual mode mappings, we use gv to return to visual mode with the previous selection highlighted after the :move is complete.

By default, MacVim maps ^⌘↑ and ^⌘↓ to move around the quickfix list. We can disable that by setting the menu option’s key to <nop>. Add this to your .gvimrc:

If you hit ^⌘↑ or ^⌘↓ with a character-wise visual selection, the whole line will still be moved.

You can map different keys for different platforms.

We can also extend this to fix indentation of the block as you move it:

This is a bit unpredictable in my experience. YMMV.

#vim #sublime
Feb 5, 2014

Vim Auto-closers Compared

I type {, Vim types } for me. Vim has a number of plugins to do this and each does the job slightly differently.

  1. delimitMate by Raimondi (484 GitHub stars)
  2. Auto-Pairs by Miao Jiang (181)
  3. smartinput by Kana Natsuno (239)
  4. AutoClose by Karl Guertin (44)
  5. AutoClose by Townk (212)

My Requirements

  1. Don’t break HTML templates that use double braces, e.g. {{name}}
  2. Create a new block easily, i.e. type { and the matching } is inserted and the cursor is positioned on a blank line between the two, correctly indented.
  3. Handle escaped string terminators
  4. Don’t break other plugins
  5. Understand that " is the comment character in Vim
  6. BONUS! Press return within a string splits line, adds delimiters and a string concatenation operator.
Feature delimitMate Auto-Pairs smartinput
HTML templates Doesn’t work with HTML filetype
New block ✔ (type {<CR to open a new block)
Handle escaped string terminators
Don’t break other plugins
Vim comment character
String split

AutoClose and AutoClose

The AutoClose plugins are much simpler than the 3 tested above. If you’re using one and happy, great. If you’re not using anything, try delimitMate, Auto-Pairs or smartinput.

The Winner: Auto-Pairs

I’m going with Auto-Pairs unless it breaks or I find something better.

I’d be interested to investigate smartinput a bit more, but Auto-Pairs gets the job done with minimal effort.

What About String Splitting?

None of the tested plugins wrap strings. One way to implement this would be to use formatexpr and formatprg, which tells Vim to run lines through a formatter.

#vim #plugins
Sep 4, 2013

At Christmas 1994 the NME carried a mixtape from the Chemical Brothers (still known as the Dust Brothers at that point). I lost my copy ages ago, but it was probably warped to shit from all the times I played it. This track was the highlight. I don’t think I’ve heard it since then. It’s insane to think that was almost 20 years ago. O.M.G. I’m practically dead now.

Edit: Lo and behold, somebody’s kindly posted the entire NME Xmas Dust Up mixtape to Mixcloud. Enjoy!

(Source: Spotify)

#music #spotify
Jun 7, 2013

Should You Move Away From Heroku To Protect Your Users?

Heroku provides an SSL add-on for encrypting traffic between browsers and the Heroku network.

However, SSL is terminated at the Heroku routing layer. Traffic is then passed unecrypted through the network to your application. That network is AWS.

A Heroku support agent confirmed this by email:

"I have confirmed with our routing team that HTTPS traffic will be terminated at the SSL Endpoint, and passed over to the runtime (where your application resides) in plaintext."

If your customers are submitting sensitive data to your app, like payment card details or personally-identifiable information, then that data is transmitted between nodes in Heroku’s network without encryption.

Threat?

Does this mean your traffic can be intercepted by other users? I’m not a security expert but I believe it’s a risk you might not want to take with customer data.

Apps running on Heroku do not have root access, but it’s possible to sniff network traffic even without root access, e.g. Sniffing with Wireshark as a Non-Root User.

Heroku’s Security policy claims that packet sniffing is not possible:

"Managed firewalls prevent IP, MAC, and ARP spoofing on the network and between virtual hosts to ensure spoofing is not possible. Packet sniffing is prevented by infrastructure including the hypervisor which will not deliver traffic to an interface which it is not addressed to. Heroku utilizes application isolation, operating system restrictions, and encrypted connections to further ensure risk is mitigated at all levels."

About a month ago, Heroku revised their networking model to improve isolation between dynos:

"Previously, network interfaces were shared between multiple dynos. […] The shared network interface […] resulted in a low grade information leak where one dyno could obtain some information about connections made by other dynos. This information did not include any customer data or other customer identifying information. But it broke the core principle of tenant isolation. With the new networking model, dynos now have fully isolated network configurations."

PCI

The PCI standard requires that sensitive data be encrypted at rest and when transmitted over public networks, but not internal networks. Heroku’s network is surely public, despite their considerable pains to make it look and feel like an internal network.

Alternatives

AWS allows traffic to be encrypted between an Elastic Load Balancer and EC2 instances via a feature called backend server authentication.

#heroku #ssl #security
Apr 15, 2013

Quantified Self

In January, Pew Research found that 21% of US adults track some aspect of their health using technology. This is part of a phenomenon called “Quantified Self” - the measurement of personal behaviours and activities, often using technology such as smartphones and web apps. I’m a self-quantifier and have been for a while. Over the years I’ve tracked a number of dimensions:

Driving Improvements

I use this data to drive improvements in my life.

For example, if I’m training for a race, I check that I’m building up my distance at the right rate in preparation. To understand the data, I visualise it using histograms and aggregates statistics. My all time distance on Runkeeper right now is 1,188 KM. I’ve only run a few KMs this month, but last month I ran almost 50 KM (I had a big race).

My self-quantification breaks down into two categories: “performance” and “habits.” The performance stuff (running and cycling) is a bit vain, but I feel that collecting and visualising performance data has helped me to achieve some big goals, like running a marathon.

Better Habits

Measuring some of my habits helps me to make long-term behavioural changes.

Meat-eating is a good example. I love meat, but I like to keep the total amount I consume down. I track whether I ate meat or not each day by ticking a box in chains.cc. Later I can look back and see easily whether I’m sticking to my goal or if I need to work harder. Having aggregate goals like “eat meat once per week” allows me to indulge when the meat on the table is really good (like when I went to Aux Lyonnais) but means that I get the health benefits of a mostly plant-based diet.

Self-Experiments

Some self-quantifiers conduct self-experiments using the data they collect, e.g. “does taking drug A make my condition better or worse.” I’m not quite there yet, but it’s an interesting direction. I’m not sure I have the discipline to collect sufficiently high-quality data.

For more about the Quantified Self movement I recommend checking out the Quantified Self blog.

#quantifiedself
Apr 5, 2013

Serve HTTPS From Elastic Beanstalk Application Instances

Elastic Beanstalk is Amazon’s platform-as-a-service built on top of EC2, S3 and other Amazon services.

An Elastic Beanstalk application consists of one or more EC2 instances running your application and a set of supporting resources, including an Elastic Load Balancer. By default, the load balancer listens on port 80 and forwards traffic to port 80 on your app servers. You can configure the load balancer to listen on 443 easily, but traffic from the LB to the app servers is not encrypted. To encrypt the traffic on this hop, you must configure your app servers to listen for HTTPS requests.

Options For Configuring App Servers To Serve HTTPS

This can be done one of 2 ways (examples specific to Tomcat, but the methods should be applicable to any app server).

  1. Use Elastic Beanstalk configuration files to install packages, create Apache configuration files, certificate and key files in the relevant locations.
  2. Create your own app server AMI with HTTPS enabled. For Tomcat, 443 forwarding to 80 (HTTPS)… How can I set up REAL HTTPS on Beanstalk outlines the required steps.

Method 1 seems preferable as it avoids the need to keep AMIs up-to-date, e.g. when security patches are released.

Using Elastic Beanstalk Configuration Files

  1. Create an .ebextensions dir at the root of your app dir or WAR file
  2. Copy in any files you want to create
  3. Create an Elastic Beanstalk configuration file to describing the required instance configuration changes

Example Configuration File

Custom commands are run before services are started. There’s no need to restart services if you’re changing their configuration (e.g. Apache in this case).

Note for Scala developers: SBT ignores hidden directories when building projects (and I can’t work out how to override that). You can use jar uf /path/to/file.war .ebextensions to insert the .ebextensions dir into the WAR file after packaging.

References

Login With SSH To Verify Changes

SSH to one of your Elastic Beanstalk instances to ensure that the changes have been applied correctly.

Initialisation Logging

The Elastic Beanstalk startup process writes any errors raised by custom container commands to /var/log/cfn-init.log. If your instance doesn’t start properly, for example services don’t start, look there.

Backend Authentication

Backend authentication is a feature of Elastic Load Balancer. It uses the public key of a certificate to verify that the backend app server is encrypting traffic with a valid certificate. This needs to be enabled for LB to app server HTTPS to work (otherwise you’ll get timeouts when making requests to the LB on port 443).

Security Group And Load Balancer Config File

A new load balancer and security group are created each time you deploy your application. Their configuration will revert back to the default, listen on port 80 only, whenever an environment is started. To avoid this, creating a second Elastic Beanstalk configuration file describing the required resource properties. E.g.

You’ll need to change this to point to your certificate in IAM, to restrict SSH access to the right IP range and to add your public key.

Customizing Environment Resources describes how to write configuration files for the other AWS resources in an Elastic Beanstalk environment.

How to write policies to manage backend authentication is described in the examples in ElasticLoadBalancing Policy Type.

#aws #elb #elasticbeanstalk #https
Mar 27, 2013

Mock To Mobile In Minutes With Codiqa, Trigger.io And TestFlight

I’m a fan of HTML for building mobile apps. Its incredibly quick and easy to build simple apps. I already know HTML, CSS and JavaScript. Lots of other developers do too, so it’s easier to find people to work with than native platforms.

When I recently started working on a mobile project, I used Codiqa to build a mockup of the app interface and generate HTML and Trigger.io and TestFlight to turn that mock into an app running on my iPhone.

Get Creative With Your Mock

Codiqa makes it easy to create app mocks.

  1. Sign up for Codiqa basic account so you can build more than 3 pages and export your mock as HTML
  2. Build an awesome mock by dragging and dropping
  3. Optional: customise your app by creating a theme with jQuery Mobile Themeroller and uploading it to Codiqa
  4. Download your mock as HTML and unzip

From Mock To Mobile In Minutes

Next, I used Trigger.io to create an app from the mock HTML and TestFlight to release it to my phone.

  1. Install Trigger.io’s tools
  2. Create a directory for your new app
  3. Run forge create (See Trigger.io’s OS-specific instructions for more detail)
  4. Replace the contents of the src dir Download HTML from Codiqa to the src dir of your new app
  5. Rename app.html to index.html so that Trigger knows it’s your start page
  6. Change your app’s config to include your package name and point to your provisioning profile (there’s an example local_config.json here)
  7. Run forge build ios to compile the app
  8. Run forge package ios to generate an IPA package file for iPhones

Distribute Your Mock App

Once you’ve followed these steps, you can upload your build to TestFlight and push it out to your testers.

To deploy apps to iPhones, you will need an Apple developer account and a provisioning profile. TestFlight have a good resource on preparing a build for distribution through TestFlight. Also, Handshake have a good overview of Testing iOS Apps with TestFlight.

Apple’s document About iOS Development Team Administration provides a wealth of info on the iOS development workflow.

#mobile #html #codiqa #trigger.io