Mount docker socket inside AWS container

  1. To your container, add a new volume
  2. Name: ‘docker_sock’, source path: ‘/var/run/docker.sock’
  3. In Storage and Logging section, add new mount point
  4. Select ‘docker_sock’, container path: ‘/var/run/docker.sock’

And that’s it. No need to give privileged access, and if you run docker commands directly from inside the container there’s no need to change IAM policy.

Associating EC2 instances with an ECS cluster

The EC2 instance is associated with a Container Service cluster using the /etc/ecs/ecs.config file on the instance, in the format ECSCLUSTER=yourcluster_name.

The EC2 instance must also have the ECS agent installed. If you create the instance using the ECS AMI this will be pre-installed (search for AMI called amazon-ecs-optimized).

This configuration can be put in the User Data field:

echo ECS_CLUSTER=your_cluster_name >>/etc/ecs/ecs.config

To find the setting on an instance that already exists: Actions -> Instance Settings -> View/Change User Data

Exact instructions for setting up the EC2 instance properly can be found here:

Running high trust apps / plugins with Sharepoint Foundation 2013

The User Profile service is not available on Sharepoint Foundation and therefore high trust apps are not supported.   However I found that after installing a high trust app it worked until restarting IIS, and after that it would generate authentication errors.  Re-uploading the app package to the app site fixed the authentication errors and allowed the app to run properly again.

I could not find an API to automate this re-upload process, so instead it can be done using a headless browser.

Here’s an example in casperjs (note that on Windows casperjs must be installed in a path without spaces):

var url = ''
var file = ''
var user = 'username'
var pass = 'password'

var casper = require('casper').create();

casper.setHttpAuth(user, pass);

casper.thenOpen(url, function() {
  this.waitForSelector('#idHomePageNewDocument-WPQ2', function() {
    this.echo("Found selector");

casper.thenClick('#idHomePageNewDocument-WPQ2', function() {
  this.echo("Clicked button");
  this.waitForSelector('.ms-dlgFrameContainer > iframe', function() {
    this.echo("Got the iframe");

casper.withFrame(1, function() {
  this.waitForSelector('#aspnetForm', function() {
    this.echo("Found form");
    this.fill('#aspnetForm', {
      'ctl00$PlaceHolderMain$ctl01$ctl04$InputFile': file,
    }, false);
    this.wait(3000, function() {'#ctl00<em>PlaceHolderMain</em>ctl00<em>RptControls</em>btnOK');
      this.echo("Clicked button");

The term ‘Get-SPApplicationPrincipal’ is not recognized as the name of a cmdlet, function, script file or operable program.

Ignore what it says on this technet page:

The command is actually Get-SPAppPrincipal.

You can see the correct command here:



Handling HTTP status code 100 in Scrapy

You might have some problems handling the 100 response code in Scrapy.  Scrapy uses Twisted on the backend, which itself does not handle status code 100 properly yet:

The remote server will first send a response with the 100 status code, then a response with 200 status code.  In order to get the 200 response code, sent the following header in your spider:

‘Connection’: ‘close’

If your 200 response is also gzipped, Scrapy might not gunzip, in which case you need to set the following header as well:

‘Accept-Encoding’: ”

And if Scrapy will not do anything with the responses at all, you might need to set the following Spider attribute:

handle_httpstatus_list = [100]

Oh noes! Kivy dependency installation has killed my graphics

If you are in the habit of blindly installing packages onto your personal machine while following installation instructions for some new shiny program, and that current shiny program is Kivy (1.9) and your version of xorg is a bit out of date, you may find a black screen the next time you boot.

This is due to the following kivy dependencies which have upgraded your xorg dependencies and broken xorg:


When you try and start xorg, you will get an error similar to:

No such file or directory: /usr/bin/x


To fix, upgrade xorg to quantal version:

apt-get install xserver-xorg-lts-quantal


I rebooted into failsafe graphics mode, then normal mode (both failed), then normal mode again and everything was working.

Windows console encoding

Filenames on NTFS are encoded in UTF-16.  The windows console is set by default to some other encoding entirely.  This makes working with files with ‘special’ characters in the filenames impossible…

In my case, I was using the following common code to delete files and folders in a directory:

set folder=”C:\test”

cd /d %folder%

for /F “delims=” %%i in (‘dir /b’) do (rmdir “%%i” /s/q || del “%%i” /s/q)


But files with certain unicode characters were not being deleted.  To fix this, add the following at the top of the file:

chcp 10000


This changes the encoding to UTF-16.

Or if you’re using cmd and the dir command, change the font first to Lucida Console (as the default font has a very limited character set).

Sharepoint 2013 Provider-Hosted App Architecture Notes

Trying to build a Sharepoint 2013 app has probably been the worst experience of my coding life so far.

The Microsoft docs make it sound so easy; there are so many ways you can build an app!  You can use any programming language you like!  Look, we have a REST interface!  Look, mobile app APIs!

Hey awesome,  you think, looking through the initial introductory documentation, yeh all the different information is a bit confusing, but look, they have how tos and the APIs are documented properly, how hard could it be?

Well, after wasting A LOT of time following guides and trying to build solutions that work, here’s some information that happened to be crucial to the architectural decision making of the apps that I didn’t come across until much too late.  Probably it’s wrong, because I’m finding it extremely difficult to get actual facts about the different ways you can build sharepoint apps, despite the millions of confusing articles on the Microsoft site (none of which seem to contain all the information you need to know), and lots of tutorials (written only by people coding in ASP hosting their sites on Azure or using OAuth).


Provider-hosted apps using the REST API:

  • You can either use the javascript cross-domain library or use OAuth
  • Using OAuth requires an account with Azure AD and you also need to configure your Sharepoint installation to use Azure AD (and obviously the Sharepoint installation needs access through firewalls etc to communicate to Azure AD).  In addition, the app needs to be registered in Azure.
  • I’ve seen some tutorials that say for testing you just need to register the app in SP and not Azure, and that you don’t need the Azure AD in this case; I couldn’t get this to work.

Provider-hosted apps using high trust:

  • The how-to guides all use a couple of Microsoft provided C# files for the authentication, in addition to Windows Authentication for the site in IIS, and I can’t see any documentation on how the process actually works.  Reading through the files, they get the Windows user information, so I have a feeling this method can only be used for apps built (1) in ASP/C# running on a windows machine, and (2) in the same domain as the sharepoint installation.


So if you want to build an app that can modify sharepoint data in any non-Microsoft language, and host it on a non-Windows machine, and don’t want to pay for an Azure subscription, and don’t want to change the authentication method of your sharepoint site, your options are:

  1. Javascript frontend to deal with Sharepoint, plus likely a backend of whatever to do anything you can’t with javascript (use 3rd party APIs etc)
  2. A high trust app to act as a proxy between your app and the sharepoint installation*

*I’m still trying to figure out how it would be possible to send the REST request I want to make to sharepoint to the proxy instead, and have that sign it and forward it on to sharepoint…