Software Engineering Blog

Feb. 13, 2017

Azure Blob Storage - Java Examples

Examples in Java for working with Azure Blob Storage

Full Article »

Jan. 27, 2017

Using ELK and Ntopng to monitor data downloads.

How we use ELK (Elasticsearch, Kibana, Logstash) and Ntopng to track and visualize data downloads

Full Article »

Jan. 16, 2017

Supporting D3 v3 and v4

D3 is a popular javascript library for data visualizations. Last year they released version 4, which had a significant rewrite of the API. This introduced a problem for us because we had one project using D3 v3 and another project using d3 v4. These two projects shared some code and therefore we needed to be able to support both versions of D3 depending on which one was available.

Full Article »

Jan. 3, 2017

Migrating a legacy frontend build system to Webpack

Migrating a legacy frontend build system to Webpack

Full Article »

Dec. 12, 2016

GitHub repository as job scheduling system to orchestrate large data transfer

The ICGC Data Coordination Centre was tasked to transfer an over 700TB dataset into cloud storage systems. We developed a simple and reliable job scheduling system based on GitHub repository, and successfully employed it to orchestrate and track the execution of over 45,000 transfer jobs to complete the task.

Full Article »