Intro
This article describes what Datrabatix is and two of the forms in which it is currently available for use as a product to Eigo customers.
What is Datrabatix?
Datrabatix (DBX) is a mix of ETL and CEP technology that has been continuously developed by Eigo since 2001 into a mature powerful tool.
ETL = Extract, transform, and load. In its most basic installations DBX is used as an ETL platform. The primary core of the DBX system was designed as an effective ETL platform and the majority of its workload remains in this area.
CEP = Complex Event Processing. DBX can be used to monitor many criteria and to automatically trigger actions on subsets of criteria being met.
DBX consists of a class library that sits on top of a set of tables which can be installed in any SQL Server database. These classes are referenced in a custom built application to make use of the powerful DBX profiles within the application. The applications could be web, service or windows forms; typically a forms application is used to continuously run a DBX scheduling class that takes care of almost all DBX activity automatically.
There is an easy to use DBX profile editing GUI tool that also allows users to document all the profiles currently in the system at the click of a button. There is a snapshot view of recent event activity within the system. Eigo is currently in the process of adding dashboard reporting tools to this interface.
The GUI tool can intelligently preview and profile delimited files and speed up import profile creation.
What can Datrabatix do?
The following is a list of default capabilities of DBX
- Moving Files
- Copy from folder to folder on network, optionally move and/or rename.
- Send and receive files via FTP
- Download files from web URLs
- POST data to URLs
- Delete files
- File decompression
- Direct table data transfer between SQL Server databases on same or different servers
- Importing Files
- Import delimited files
- Import Excel files including multiple sheets
- DBX can handle inconsistent files by where columns have changed position or rows of none data are mixed with data rows
- Reporting
- XML datasets
- HTML formatted reports
- Delimited (by anything) files
- Send emails with data imbedded or attached
- Decision Making
- Perform actions based on sets of event criteria
- Scheduling
- Perform actions at specified intervals
- Perform series of actions in batches
- Trigger custom actions to be handled in parent application and receive and act on information on the success of that action
- Processing Data
- Any data manipulation that is possible within SQL Server can be performed from within DBX
This list is usually expanded on with custom functionality for each DBX installation as normally each customer has some unique requirement that is not part of a standard DBX install, for example uploading batch orders to SAP or receiving data feeds from barcode scanners.
DBX is not normally used as a complete system on its own but rather the workhorse for a system to continuously maintain, monitor and report on millions of records.
Case Study : FlyWorldWide
FlyWorldWide is a flight and hotel consolidator that uses DBX to manage all data imports from suppliers and exports to customers. Data from a number of suppliers come in very diverse formats ranging from simple comma and fixed width delimited files through to complex extensible proprietary EDI formats.
DBX schedules run to actively collect data from suppliers at specified intervals or are triggered to begin collecting data by some other events happening. DBX monitors FTP folders to wait for new files to arrive from certain suppliers and then automatically decompresses any which are in an archived format (eg zip or gz) and imports the files.
The data files from the various suppliers are converted down to flat data and missing fields calculated or split from other fields. Data is converted to the correct data types and finally all supplier data is consolidated into a single data table of all flights from all suppliers.
FlyWorldWide use this data to drive an XML booking interface which is not part of DBX and also a flight matrix for their telephone support users to rapidly view flight data.
DBX is also used by FlyWorldWide for automated reporting on KPIs and data currently in the system. DBX also outputs reports in XML, formatted HTML and CSV files for consumers of FlyWorldWide’s services. These reports are triggered either from schedules, user triggers or when specified criteria are met.
Case Study : Cobra Project Management
Cobra Project Management is a tool used in house by Eigo to run all of its own and customer’s projects. DBX is used to monitor projects, deliverable and tasks, automate reporting to customers and project sponsors, to perform billing and SLA KPIs.
How is DBX available to customers?
DBX is not usually sold as a product on its own but used to deliver solutions on large customer projects. However with only a small amount of consultation and configuration we can quickly deploy a DBX implementation.
DBX is currently only available as a system managed by Eigo under consultancy for the customer, this can be hosted in the Eigo data centre or on the customer’s own servers.
Steps in setup of DBX for client:
- Eigo consult with client on what their requirements are, this is usually FREE
- Eigo engineers install DBX on the customers servers or set up a new hosted version in our data centre.
- If required Eigo build applications and processes to support unique customer requirements defined in step (1)
- Eigo build standard DBX profiles to perform the businesses processes defined in step (1)
In future we plan to offer DBX as a packaged product for customer management either hosted in the Eigo data centre or on the customer’s own servers. To this end we are working on training packages, user manuals and improved user interfaces and tools.
A lot of value of the product lies in the expertise and experience of the Eigo engineers hence for the majority of customers the Eigo managed solution is the right choice.
How do I find out more?
If you are interested in DBX please contact us or submit a consultancy request.
DBX GUI Screen Shots