Introducing the Modern Manufacturing Framework

Manufacturing is responsible for the clothes you wear, the products you use every day, and has assembled nearly every object in the room with you right now. Data, data, data - Manufacturing is responsible for generating Exabyte's of data that gets collected, stored, and analyzed every year. That's far more data than any other sector.

Plant Worker

In my role with Microsoft, I'm working with commercial software vendors on a daily basis to help them build cloud solutions. Manufacturing is becoming extremely competitive, to the point where the only survivors are those that can efficiently mine the insights from their data in real-time and adapt/react quickly. Thanks to technology, manufacturers in the United States have become the most productive in the world.

Within the Technology Evangelism & Development (TED) team, we're working hard to identify common patterns where we can build reusable open source frameworks. My goal is to bring these frameworks together in the context of manufacturing, while filling in some of the holes that exist currently. Microsoft already has a reference architecture for Discrete Manufacturing (DIRA). While the goals of the DIRA project are similar, our project complements those patterns with a concrete implementation showcasing Microsoft technologies.

Manufacturing Trends

Data collection in manufacturing is rooted in technologies whose protocols were developed decades ago. Traditionally, data collection and storage has been siloed by physical location. Companies with multiple facilities have struggled with not just getting their data into a central location, but storing and processing that data at scale. Cloud computing can provide a centralized storage location and the scaleable processing required to make sense of it.

When you look at the design of software that has been around for a decade or more, you'll see that as an industry we've gotten really good at adding features. Historically this meant adding more buttons to toolbars or menus. Modern software design requires us to focus on usability. In manufacturing, this means designing intuitive role-based displays. Mobile software has taught us that focused simplicity can be a valuable advantage. Less training means lower costs for new employees, and makes it easier to collect the right information from the right people.

Toolbars vs Simplicity

This framework is...

  • focused on the MES portion of discrete manufacturing.
  • an end-to-end data pipeline capable of pulling data from existing systems (using adapters) and ultimately display that information and provide self-service business intelligence.
  • decomposable - components are interface-based so that any portions can be used individually. Use as little or as much as you like.
  • extensible - because of the modular design approach, the framework can be extended limitlessly.
  • open source - it will be licensed under the MS-PL.
  • using Microsoft technologies such as Windows Azure and Windows 8, although non-Microsoft technologies will be used where appropriate.
  • aligned with tomorrows manufacturing trends such as Industry 4.0.

This framework is not...

  • competing with other Microsoft efforts. It's meant to fill in gaps, not replace existing solutions in development.
  • the best way to push data. It simply demonstrates one possible solution.
  • the best way to store data. It simply demonstrates one possible solution.
  • The best way to process data. It simply demonstrates one possible solution.
  • embedded. This framework is a level above the embedded device ecosystem, but can use data generated or collected by devices.
  • competing with partners already in this space. This is designed to help accelerate partner application development, and gives them opportunities to add their business value.

Going Forward

For now, my goal was to simply introduce the project. I'll be working on a series of blog posts discussing the architecture and the various goals. Be sure to subscribe for updates!

Like this post? Please share it!

See a mistake? Edit this post!

7 Lessons from Running a Hackathon

I'm working on a new project that will demonstrate how the current and future trends in manufacturing can be enabled by Windows Azure and Windows 8. I'll be sharing more information in the future.

We decided to have a weekend Hackathon with Skyline Technologies to kickstart the project. I recently had someone ask me a few questions to help run these in the future. I decided to write this up to reach a wider audience.

Now, there may be some confusion as to what a Hackathon actually is. The compendium of human knowledge suggests that they come in all shapes and sizes. In my particular case, the goal of ours was to dive in and start writing code without being slowed down by process or restrictions, allowing the ideas to flow.

A Hackathon (also known as a hack day, hackfest or codefest) is an event in which computer programmers and others involved in software development, including graphic designers, interface designers and project managers, collaborate intensively on software projects.

Hackathons typically last between a day and a week. Some hackathons are intended simply for educational or social purposes, although in many cases the goal is to create usable software. Hackathons tend to have a specific focus, which can include the programming language used, the operating system, an application, an API, or the subject and the demographic group of the programmers. In other cases, there is no restriction on the type of software being created.


Now that some time has passed and the dust has settled, I can look back and see what made this successful, and how we could have improved.

1. Meet 6 Weeks in Advance

It's good to have your first meeting well in advance of the Hackathon if possible. Each pre-meeting will add additional definition to the hackathon plan.

This is the time when you need to start signing people up and getting commitments that they'll be there. This is also a good time to talk about the high level goals of the project. The goals and the participants will start to align early if well executed.

Ideally, you'll have a mix of devs, preferably with diverse backgrounds. Also keep in mind what subject matter experts will be most helpful for your requirements.

2. Meet 4 Weeks in Advance

4 weeks in advance, start confirming some of the details. Any last minute developer changes? What are the exact places and times that people should show up? Start talking about what will actually be delivered as a result of this hackathon, and how that matches the commitments.

3. Create Your Solution & Dependencies

In my experience, every Hackathon starts with an hour or more of people getting their machines configured properly. Getting their system up-to-date, connecting to source control, pulling the source, and installing dependencies. Not only does it waste precious time, it immediately drops productivity.

I've seen this problem solved with virtualized development environments. You create a reference image, and then each person copies and runs a clone of the image. This speeds up the process, but still doesn't get us past issues with permissions, pulling the source, and building.

The best solution I have found is to set up the solution ahead of time, referencing all of the required dependencies. If the solution needs the Windows 8 SDK, include an empty Windows 8 project. If it needs Azure components, create a cloud service. In your pre-hackathon meeting, make sure that everyone knows they should have the solution building before showing up for the event. By setting the expectation to be ready ahead of time, we can avoid the time consuming process of downloads and installs.

Setting up the solution ahead of time also has the added benefit of avoiding the merge conflicts that are more likely at the beginning of a project.

4. Meet 1 Week in Advance

A week before the hackathon, this is your last chance to tie up any loose ends. This is also a good time to talk about prerequisites, so that everyone can get their development machines properly setup.

5. Have a plan for food

People need to eat. Provide lots of options, and make sure that food is timely and plentiful. Make sure there are no special dietary requirements. You'll also want to make sure that there are a variety of drinks available, so that people can stay hydrated and caffeinated.

6. Network Switch & Other Hardware

The rule in my house is that if a device doesn't move, it must have an Ethernet cable run to it. Wi-Fi is only for devices that move frequently such as phones, tablets, etc. When it comes to video calling, sharing files, and Internet access, there is simply no substitute for a hard-wired connection. Have plenty of switches and cables available so that everyone gets a port.

You should also have extra monitors, USB hubs, mice, keyboards, and lots of various cables on hand. Have plenty of whiteboards with working markers, and a projector that anyone can hook up to. You want to make working and sharing as frictionless as possible.

7. Have Someone Present that is Accountable

People who attend hackathons show up ready to work. If they didn't want to be there, they would probably have stayed home. That being said, there should be a designated person whose primary goal is for the overall project to be successful.

There will also be lots of decisions that need to be made, and not a lot of time to make them. Discussions should occur, but often times making a bad call is worth it to keep things moving. Having someone that can provide answers is critical.

Have fun!

Like this post? Please share it!

See a mistake? Edit this post!

Convention-Based Configuration File Loading

I'm working on a modular project with numerous services and processes that all work and communicate together, which requires a fair amount of configuration. In my IoC configuration, my code was starting to get ugly due to the need to manually inject constructor parameters for things like port numbers, timeouts, addresses, etc.:

    new TransientLifetimeManager(), new InjectionConstructor(dataCollectorQueueName, typeof(IDatasourceRecordSerializer)));

Every class that required configuration (in this case, dataCollectorQueueName) required an explicit registration line since the parameters are ambiguous to an IoC container. I use IoC containers as a productivity tool, and they have very little value unless I can automatically wire-up dependencies by convention.

I created a method that loads a JSON file and maps it to a class instance at run-time based on a specified naming convention. You can include the configuration files in your project, or store them anywhere you like. In my case, I've included them in my project and set them to copy to the output folder. When the code is used in production, the files can easily be edited. This could even be extended to watch the files at runtime and make configuration changes on-the-fly.

Here is the code:

public static void LoadFromConfigurationFile(IUnityContainer container, string file, string classNameConvention)
    var fileName = Path.GetFileNameWithoutExtension(file);
    var className = string.Format(classNameConvention, fileName);

    var types = new List<Type>();
    foreach (var assembly in AppDomain.CurrentDomain.GetAssemblies())
            var t = assembly.GetTypes();
        catch (ReflectionTypeLoadException ex)

    var matchingTypes = types.Where(x => x != null && x.Name == className).ToList();
    if (matchingTypes.Count == 0)
    if (matchingTypes.Count > 1)
        throw new Exception(string.Format("Multiple types found with the name '{0}'", className));

    var type = matchingTypes.First();
    if (container.IsRegistered(type))

    var configJson = File.ReadAllText(file);
    var deserialized = JsonConvert.DeserializeObject(configJson, type);

    //Naming the instance BREAKs the resolving
    container.RegisterInstance(type, deserialized, new ContainerControlledLifetimeManager());

Here is process:

  1. Read the name of the file. This is important for our convention.
  2. Using the classNameConvention, determine what class name we're looking for. I'm using "{0}Configuration".
  3. Load all available types.
  4. Find the class with the name that matches our convention.
  5. Deserialize the configuration file into the type we're mapping to. I'm using JSON.NET.
  6. Register the object with our IoC container. I'm using Unity.

Then, to turn this into a one-liner in my application, I created a method that iterates through all the files in a folder, mapping their configuration data by calling our original method.

public static void LoadConfigurations(IUnityContainer container, string configurationFilesPath, string classNameConvention)
    var configurationFiles = Directory.GetFiles(configurationFilesPath);
    foreach (var file in configurationFiles)
        LoadFromConfigurationFile(container, file, classNameConvention);

Here is what a configuration file can look like (DataCollector.json):

    "Port": 1234,
    "MsmqQueueName": "DataCollectorQueue"

And the corresponding class. Notice the name follows our convention.

public class DataCollectorConfiguration
    public int Port { get; set; }
    public string MsmqQueueName { get; set; }

Now, any class that needs configuration data from this class simply requires it in the constructor:

public class Foo
    public Foo(DataCollectorConfiguration config)
        //We have our configuration here!

Application Configuration

The auto-registration that Unity (and other IoC) provides will auto-wire configuration classes to any classes that need it. Very little code is required:


This makes creating new classes simple and fast. We just define what our dependencies are in our constructor signature, and we allow them to be supplied to us. That's the core idea behind dependency injection. No need to worry about both the consumer and producer of the object instances. We can focus on the logic of the class we're creating.

Problems in Paradise

I ran into a fairly significant issue/bug with Unity. After registering my configuration classes, the auto-registration in Unity would overwrite my configuration instances. Unity is actually supposed to throw a DuplicateTypeMappingException when auto-registration is trying to register a class that is already registered. In short, Unity was allowing me to shoot myself in the foot.

The solution was to filter the type list that Unity was using. First, I created a testable extension method that would allow me to filter a type list based on an existing type list:

public static IEnumerable<Type> ExcludeTypes(this IEnumerable<Type> applyToTypes, IEnumerable<Type> excludeTypes)
    return applyToTypes.Where(x => !excludeTypes.Contains(x));

I was then able to switch the auto-registration code from AllClasses.FromAssembliesInBasePath() to AllClasses.FromAssembliesInBasePath().ExcludeTypes(registeredTypes):

var registeredTypes = container.Registrations.Select(x => x.MappedToType);



I wrapped up my configuration blocks in classes, making the entire system modular and easy to use. Now I can focus on adding code to solve business problems, not worrying about wire-up. The code I use to build a new service in the application simply has to initialize its configuration using code like this:

_container = new UnityContainer();
ConfigurationLoader.LoadConfigurations(_container, ".\\Configuration\\", "{0}Configuration");
//Register any other modules needed for additional functionality

Like this post? Please share it!

See a mistake? Edit this post!

Jason Young I'm Jason Young, software engineer. This blog contains my opinions, of which my employer - Microsoft - may not share.

@ytechieGitHubLinkedInStack OverflowPersonal VLOG