We are Data Engineers
Bringing the Future of Automated Data Management
Turning data into usable information has always been important. More than ever before this means automating the process and doing it on an industrial scale. As data engineers we help you achieve this, allowing your business to work at a new level. We use a combination of different tools to build a firm foundation for your business’s digital transformation.
Our Tools:
Harvesting Growing Cleaning Integration Transport
Harvesting
Harvesting is an automatic process that makes sure your data is there where you need it when you need it.
A typical bank might have 1000 external files per week, say in the form of email with an attachment.
Our Harvester uses php and sql to strip out the attachment and shake out the data. Then it’s ready for cleansing.
Growing
Once all the data is in a common structure the data really starts to work. In the case of a medical facility this structure can save lives, because it might be checking the potential drug conflict of prescription medicine, before administering it. This added value is what we call growth.
Cleansing
Someone’s sent you a big file, but out of the 30 columns on the spreadsheet attachment there are only 18 you need. Cleansing takes the spurious stuff, checks things like zeros haven’t become the letter O, date formats haven’t become American etc. Cleansing is what makes the data usable. After all the famous maxim rubbish in, rubbish out, and who wants rubbish?
Integration
Integrating is getting data into a common useable structure, despite very different origins. Imagine you have to take ice cream sales from your tills you might want to integrate that with hours of sunshine or weather data from the met office. Integrating is bringing these two together.
Transport
For all the processes that have to happen, data needs to be moved to where it is going to be used and be most useful. Obviously, it is important that this is done in a way that preserves the integrity of the data, using the most appropriate vehicle and protocols. It also needs to be done robustly and reliably, without the operations missing a beat; transport is crucial yet often overlooked.
Hover over the icons above to discover more about each process
It is all about Automation
We help organizations become proficient at working with data. This means creating automatic systems that process the large volumes of data from a wide variety of sources. There’s no one size fits all process, we design it all to deliver the results you need to achieve.
A better way to drive the business
A strong automated data management system is a little like an autopilot. It’s doing a lot of the day to day work for you.
But far from making the C suite redundant, it frees you up to see the big picture, do the decision making and plan ahead.