Over the last few days I’ve been working on interfaces to use across multiple programming languages to standardise how some things are done. This stemmed from a requirement to allow processing to be dealt with in any language from any other language. People do this all the time, it’s nothing new, but they generally stick to “language A will have a requirement to get some data and we’ll get this data using language B across multiple servers/threads”. This results in an API contract between the two languages but not much else (you, in advance say I’ll request it in this way and receive it in this way) but it’s often a unique way and a newcomer wouldn’t necessarily understand it immediately.
What I want is a way to know, in advance, what to expect between any language, a schema if you will to say “I will adhere to this implementation of parallelisation and this method of data transfer.” I am working towards that goal and so far it is going well. What I do not want to do is restrict people however which means the implementation still has to be abstract enough to work for all situations but concrete enough to make doing it in any language a breeze.
For the parallelisation I am using a form of events, event handlers and event input and output interfaces. These allow for a guaranteed method of delivering and receiving a description of the work to be done but the flexibility to deliver any relevant data. I intend to expand on the interfaces for common multi-threaded tasks in order to standardise as much as possible in any many languages as possible – something I will undoubtedly need help for.
With regards to logging (which will have an example using the parallelisation as an extreme example) it borrows from the event driven interface but standardises the log format, a simple but essential requirement. The log levels are that of syslog and the rest of the data allows it to be used in a simple, single server, app to a distributed system. The key here is every language logs in exactly the same way and can be off-loaded into the event system of the parallelisation interfaces who can handle it in any language as they will all have a copy of the contract (interface). To see just why this is needed, search for things like “php logging”, “c# logging”, “python logging” and compare just how different languages do the fundamentally same thing. And then try and let an application in PHP log using a system written in C# – it just becomes more custom code.
The transport between languages will be doing using JSON due to it’s relative ubiquitous nature among all modern development languages. Rather than just say “use JSON” however I am implementing schema’s for the JSON data using the existing (but unfinished) standard being developed by json-schema.org. This is one of the biggest part of this venture, not only to say “adhere to this interface and we can work together” but to say “adhere to this interface and you can be 100% sure you can validate me against this public schema, regardless of the language you want to use”.
For now, you can see the work in progress at https://github.com/xcitestudios. They will be open source and I welcome feedback.
I am currently working on examples in PHP and C# for the interfaces to show why they exist and how they work. The logging interfaces are self explanatory, but the parallelisation interfaces will require some examples to explain just why they exist.
Aside from examples, Media Skunk Works will be using these interfaces (alongside proprietary work) to help scale work extremely cost efficiently. That is some time off though; but will prove their value.
