Page maintained by
The ISDAT Instrument Measured Database Management SystemThis section will probably be rewritten in the near future, I feel it does not provide a developer with a good start RG.
BackgroundThere exist numerous types of instruments which are the basis of experimental science. Typically, the instrument is capable of storing data in some format which is suitable for that instrument. We call such a format the raw instrument format. The raw instrument format is typically not self describing to humans. To make use of the stored data from some instrument, it is necessary to transform this raw data into some self-obvious representation such as a plot or a table. We call such a format human interpretable format. Such a transformation from the raw instrument format to the human interpretable format must be developed. ISDAT autonomously provides an human interpretable data interface to raw data but as to standard data formats.
PurposeThe purpose of ISDAT is to allow a high-level access to stored raw data acquired from arbitrary instruments.
Main MotivationThe main motivation for ISDAT is based on the fact that
What ISDAT is not
The Philosophy of Measurement and uniform data accessTo acquire knowledge of physical reality it is necessary to partake in measurement acts. The results of natural science must ultimately depend on measurement acts consisting data collected by (non-subjective) instruments. The philosophy of science has come to an understanding of such measurement acts. Within the modern model, a measurement act is associated with a certain region within the framework of space and time. Measurements must ultimately be attributable to, what in the theory of relativity is known as, a WorldLine or a collection of WorldLine know as fiber bundle.
ISDAT uses the concept of the WorldLine to organize the reference to all data within an ISDAT database. This provides a uniform and powerful way of accessing any type of instrument measured data which is archived according to this philosophy.
A detailed analysis of WorldLine viewpoint of data access reveals that it is not particularly useful for describing certain types of scientific data such GIS (even though it is possible to encode some GIS information within this model) and ironically very amenable to economical data such stock market indexes (as they are associated with a measurement at a certain time at certain stock market exchange). However it is intuitive power of this referencing method which is important and not a discriminating definition of its end usage.
ISDAT database logical schema (Relational Db model)ISDAT can be view according to the entity-relationship model which is the nowadays dominating model for describing databases. The E-R model is however not very amenable to the database model of ISDAT because of its highly arrayed structure and therefore not useful in practice and is for illustrational purposes only.
Client/Server architectureISDAT v2 is built according to a client/server architecture with a typical 2 tier structure. In contrast, ISDAT v3 can be represented as a 3 tier system. The FormatInterpreters, the TransactionManager and the DataConsumers. The TransactionManager and the FormatInterpreters together make up the server side of ISDAT which is called the DataBaseHandler (DBH). DataConsumers are the client side of the ISDAT client/server dichotomy.
There are two kinds of clients the dedicated client is shipped with the ISDAT software and the term ISDAT client refers to any software which can correctly use the ISDAT ClientServices interface.
The conceptual interfaces of ISDATTo the outside world, the ISDAT is defined through its logical interfaces. A developer or programmer wishing to employ ISDAT need only consider either one of two levels of interface. These two levels are. They are:
The DumpData is a minimum data access method. It is essentially interfaces a one-pass parser for that particular data-format. A one-pass parser takes a file or datasource instance and returns a stream of arrayed data.
To give a language which provides access to the raw data but which, conceptually speaking, is on scientific or physical data level, ISDAT provides the entry-level interface:
One of important features is that it enforces file or data source (an ODBC term) hiding. At the science level the actual storage of data is not important only the WorldLine referencing is needed. Besides making it closer to scientific queries it also facilitates systematic searches through whole databases.
The CientServices has besides it minimum set of atomic methods, a expandable set of derived client services which maybe named in the ListServices method. A server may thereby provide e.g. specially designed pattern recognition algorithms to be used by clients to find particular events. Other important derived client services methods will include transformation operators, e.g. ISDAT WorldLine data -> HDF, CDF, netCDF.
In practice the Logical interfaces are implemented within some environment which enables distributed access. For the moment only two implementations exist
The approach is therefore to provide a minimal query language for all aspiring new formats combined with ensured, well known and uniform high-level services of the ClientServices interface with a syntax amenable to a scientist.
OOP objects of ISDAT interfacesThe ISDAT objects are actual building blocks of the ISDAT source code and programming interface. They are rigorously defined concepts which make up the vocabulary of Im_data transaction.
User InterfacesUser interfaces are interactive clients and therefore utilize the ClientServices interface.
Graphical User Interfaces (GUI)ISDAT will provide the following GUIs
Promt-based User InterfacesWill be provided in the ISDAT v2.7 and include
Application Programming Interfaces (API)To develop code which either implements or utilizes the ISDAT logical interfaces the recommended procedures is to program using the ISDAT APIs which exist for numerous programming environments/applications:
FormatInterpreters generation using Data Format Description Language (DFDL)ISDAT will in the future implement parsers for DFDLs such as EAST. DFDLs describe the structure or format of raw data possibly all the way down to the bit level. What ISDAT will provide is promt-based tool which takes DFDL file and a project description file and then generates c++ code which can act as a FormatInterpreter and can be compiled along with the rest of the ISDAT server code.
Naturally, the motivation for using DFDL and the ISDAT DFDL FormatInterpreter generator tool is that it
Main advantages/features of ISDAT