-
[RESOLVED] Unknown Database Format
So I have the old db system (accessed with a terminal emulator) that I need to export the data out of but dont know what the back-end is using. Its comprised of 3 files per table it appears. a *.S, *.I, *.D
Figure maybe someone tried to get job security by renaming the extensions perhaps. opening them with a hex editor it looks like the S is a structure file (table definition), the I file possibly an index file and the D file for the data.
Any reputable website recommendations where I could upload the files to for analysis would be great. Tried a few and always fail to come up with an answer.
Thanks
-
Re: Unknown Database Format
I should add that the goal here is to export the data out and into sql server.
-
Re: Unknown Database Format
Could you post the first 100 bytes (or how many ever) of each file?
-
3 Attachment(s)
Re: Unknown Database Format
Ok I found a set of files that shouldnt contain any sensitive info. Looks like the main menu
-
Re: Unknown Database Format
By viewing the file properties I get "Preprocessed C/C++ Source (.I)"
By going online using this tool: https://www.aconvert.com/
I get (drum roll) .TRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRR.......
"Congratulations! You found a file that isn't one of the 8000+ file formats that we can recognize." :lol:
-
Re: Unknown Database Format
Quote:
Originally Posted by
sapator
By viewing the file properties I get "Preprocessed C/C++ Source (.I)"
By going online using this tool:
https://www.aconvert.com/
I get (drum roll) .TRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRR.......
"Congratulations! You found a file that isn't one of the 8000+ file formats that we can recognize." :lol:
Yea the I extension shows in Windows as a C/C++ file but I feel the extension has been renamed for its original database file extension.
-
Re: Unknown Database Format
I've looked at the files, and can't make heads or tails of it.
As for renaming the file-extension: How is the client supposed to be working, if the client looks for, say "04.idx", but it's renamed to "04.i"?
I don't even see a big difference between the D and I file. Yes, the S-File is pretty short and compact.
To me, the D-File looks like a ressource-file for a Menu in a UI. Just open it in pure notepad, and you'll see what i mean
-
Re: Unknown Database Format
Yes I first opened the files in a hex editor and notepad++ but figured nothing more than the S is the possible table definition. Its a very old system from the 80s at least.
I'll upload some larger files in a few if I can that dont have sensitive info in it if I can find them.
-
Re: Unknown Database Format
The files (at first) look like old DBase (.dbf) files, but after checking in hex viewer and the dbf files structure reference, they are not. Other memories I have are about old Paradox (made by Borland) but quick searches didn't helped me to find file structure and compare how bytes "fit".
-
Re: Unknown Database Format
Hi,
the extention .i looks like the old -Intermediate file from Borland
have a look here if it's a Valid extention you'll find it here
https://www.webopedia.com/quick_ref/...nsionsfull.asp
-
Re: Unknown Database Format
Just tested with two Paradox viewers for .db files (first renamed files to a/b/c.db) and no luck - all are invalid according to the viewers. One of the apps is using Borland database engine so it ensures that these are not Paradox files.
-
Re: Unknown Database Format
The system is running on a Linux box if that helps.
Quote:
Originally Posted by
ChrisE
Link doesnt work and couldnt find it from the main page
-
Re: Unknown Database Format
Hi,
the Links works from here.
you can try and open the file with this https://filext.com/file-extension/BORLAND
if it's a BorlandFile, hope it works
-
Re: Unknown Database Format
Quote:
Originally Posted by
ChrisE
Ok that link worked. It came back with "It's a text file, but the file format is not in our database."
-
Re: Unknown Database Format
Quote:
File name: 5003.D
File type: binary file
File size: 53 MB (1% are shown)
File date: 7/9/2019, 2:14:28 PM
The file format could not be recognized.
Seems the D and I files are binary and the S file is text
-
Re: Unknown Database Format
Quote:
Originally Posted by
RobDog888
Seems the D and I files are binary and the S file is text
pitty, well you know now it's not a Borland File Format, I'll ask someone I know,
he's retired by now I think, I'll let you now if I have any luck
-
Re: Unknown Database Format
After looking at the .main executable file in a hexeditor it appears that it contains all the fields which may just be class properties BUT also found "billing_billto.idx" which I assume is an index specifier on that billing table
-
Re: Unknown Database Format
Quote:
Originally Posted by
RobDog888
After looking at the .main executable file in a hexeditor it appears that it contains all the fields which may just be class properties BUT also found "billing_billto.idx" which I assume is an index specifier on that billing table
that's helpfull -> .idx
was used buy
- dBase
- Visual Foxpro
- Informix
-
Re: Unknown Database Format
Found "C-Tree(tm)" in the hex. Googling now
-
Re: Unknown Database Format
you can also google for elevatesoft they have "Additional Software and Utilities"
I think it's called dbsys.exe for opening the Files
-
Re: Unknown Database Format
Couldnt find the file on the site and probably have to create an account to download a trial version
I did find this which is very interesting.
https://docs.faircom.com/doc/ctreeplus/74299.htm
I think this may be more of what its set up as
https://docs.faircom.com/doc/ctpodbc/37896.htm
Perhaps I can connect to a file using an appropriate odbc driver
-
Re: Unknown Database Format
RoboDog, the links are explaining about encryption mostly, but the files you provided in hex/text view are looking that they are not encrypted:
https://asmlair.net/pics/vbforums/cr...b_hex_view.png
If someone is interested, this view is from Far Manager, a clone of the (very old) Norton Commander, but now it is 64 bit Windows app, which is updated nowadays and is open sourced.
Edited:
Some questions:
- So the db is running on Linux, created in 80s (pre-Linux era), there are Windows clients?
- How data is accessed: via client-server connection like SQL queries or another API?
- Old way (as I have seen during years) to access database files is to grant network shares (for Windows apps) that can access these database files. Is it something similar (as opposite to SQL-like or another API queries)?
Hope these questions help getting more info on the problem.
-
Re: Unknown Database Format
Quote:
Originally Posted by
peterst
RoboDog, the links are explaining about encryption mostly, but the files you provided in hex/text view are looking that they are not encrypted:
http://www.vbforums.com/
If someone is interested, this view is from Far Manager, a clone of the (very old) Norton Commander, but now it is 64 bit Windows app, which is updated nowadays and is open sourced.
Edited:
Some questions:
- So the db is running on Linux, created in 80s (pre-Linux era), there are Windows clients?
- How data is accessed: via client-server connection like SQL queries or another API?
- Old way (as I have seen during years) to access database files is to grant network shares (for Windows apps) that can access these database files. Is it something similar (as opposite to SQL-like or another API queries)?
Hope these questions help getting more info on the problem.
Its not encrypted as I have been able to read most of the data and determined which are teh Structure, Index and Data files but am unsure if they camouflaged it was my main point in posting the link..
Yes its running on a Linux server (since the early 00's was when the server was upgraded from Unix I believe).
The data is accessed via a terminal emulator session from various windows clients
No file sharing AFAIK
Appreciate teh help! Im going to look at Far Manager now, Thanks
-
Re: Unknown Database Format
Can you show some examples how it is used from the terminal emulator? You can hide sensitive information by replacing with something irrelevant to the queries and result or just mask it in Paint or whatever you use,
-
1 Attachment(s)
Re: Unknown Database Format
Here's a shot of the Order menu page
-
Re: Unknown Database Format
It is hard to tell from the terminal window screen capture what software might be running. Is there any chance of getting a file / directory list of what is actually on the server's file system for that database? That might give us a clue.
-
Re: Unknown Database Format
We created our own database formats in the 70's and 80's - actually into the early 1990's for Digital Equipment PDP's and VAX's.
Our own INDEX files, our own sort-pointer files. Our own schema files and database formats. And all of it ran in the functions linked directly into the executables - no server software to find that might help you figure out what the database was.
Can you change a field or two in that screen - save the record - and then look for differences in the file after that change. That will point you in the direction of where the data lives. Might be a good first step to take...
Can you find any UI's that allow you to work the schema at all? Maybe look around for UTIL or UTILITY folders?
-
Re: Unknown Database Format
Quote:
Originally Posted by
jdc2000
It is hard to tell from the terminal window screen capture what software might be running. Is there any chance of getting a file / directory list of what is actually on the server's file system for that database? That might give us a clue.
Its not really that kind of a setup. Its a custom linux *.main executable accessed via any terminal emulator.
-
Re: Unknown Database Format
Quote:
Originally Posted by
szlamany
We created our own database formats in the 70's and 80's - actually into the early 1990's for Digital Equipment PDP's and VAX's.
Our own INDEX files, our own sort-pointer files. Our own schema files and database formats. And all of it ran in the functions linked directly into the executables - no server software to find that might help you figure out what the database was.
Can you change a field or two in that screen - save the record - and then look for differences in the file after that change. That will point you in the direction of where the data lives. Might be a good first step to take...
Can you find any UI's that allow you to work the schema at all? Maybe look around for UTIL or UTILITY folders?
You hit the nail right on the head. As mentioned we have a linux *.main executable that when viewed in a hex editor we can see the links to each database file. The *.I appears to be *.idx though but probably just renamed.
Sent you a PM
-
1 Attachment(s)
Re: Unknown Database Format
So I found a couple interesting files in the root which were not numbered like the rest. Seems they are system setup files. Dont think I can post them but heres the *.P file. I believe its mapping the data (*.D) files/tables and defines the related indexe(s) for each table. I dont see where the *.S siles relate to yet though.
See the files in post #4 where they are the menu table/file and in my screen shot 4.D is listed with "_menu_del_flag"
-
Re: Unknown Database Format
I'd guess that was a print out of the schema
Very helpful!
-
Re: Unknown Database Format
Quote:
Originally Posted by
szlamany
I'd guess that was a print out of the schema
Very helpful!
Yes it it! I feel Im on the right track but I need a solid finding by Monday lol
I sent you an email a few mins ago btw
-
Re: Unknown Database Format
I'm sorry that I really can't help identifying the database, but I can explain what I know, I've seen and I think:
- Some of our customers were using very old CAD system developed in early 80s and running on Unix. CAD systems usually generate a lot of data which is used by other systems like ERP (in this case MRP - mechanical resource planning) and data was written in databases (files representing tables very similar to what DBase did at that time). I haven't seen these databases but I have seen exports to text files which we imported into our system many times and they are just simple human readable form of the binary data (e.g. protobuf's compared to json's)
- According to all the discussion above, files, screenshots, etc., during these years (80s and early 90s) many companies were developing own file formats not compatible with anything else, including database files
- If the files are having undocumented format, it is still possible to make simple reader that could read them - the file structure doesn't seem too complex and the fields information is already known (it is visible in the Unix/Linux app so each field can be evaluated what type it is instead of digging into the file binary structure)
- So creating few simple routines that read the data only from single file (indexes are useless since everything will be transferred into much more useful database form), fill inside list of objects, view in form so humans can check if it is the same like in the original one. If single data file is parsed correctly, then try with other data files and fix other problems
Usually each table is stored in separate file - even nowadays many databases do that. Parse data files, skip indexes, find some relation with other file types (e.g. D to S files) by checking the UI of the old application. Field names are visible, field types types are not important because you can recognize from the app UI or just by the name, field length (important when storing in binary files) is again not so important if you can compare how the app is viewing that exact data.
This is the way I will go for such problem. Sometimes it could be very time consuming but for old systems it could be the only approach.
-
Re: Unknown Database Format
Thank you for the reply. I think may be based upon one of the older systems but tweaked which is why its not being recognized by any interpreter
-
Re: Unknown Database Format
I have figured several things out and have concluded its a C-Tree database system and probably modified from there.
Thank You Szlamany for your help with the byte reading. I figured a faster way to read it all in and how to format is just like the hex readers do. :D
Still I dont need to recreate the wheel, only read in the data, indices are irrelevant as I only need the data in a read only static mode for reporting etc
Time to go to bed as have to get up for work in only a couple hours.
http://www.robertromo.com/ParameterFileSetupCropped.png
-
Re: Unknown Database Format
I think I may have cracked this nut!!! :D *DOIN THE HAPPY DANCE*
This is the "Court" table "D" file.
http://www.robertromo.com/5037.D.DecodedCropped.png
-
Re: [RESOLVED] Unknown Database Format
So figured out the *.S files are the field definitions of their corresponding *.D files, fixed length records, table header size which tells me where the data actually starts. Allot of detective work but looks like I'm moving on to writing the program to read it all into SQL next. No point in spending time of the *.I index files as they are useless because we will be setting up new ones.
Thanks guys!
-
Re: Unknown Database Format
Quote:
Originally Posted by
RobDog888
I think I may have cracked this nut!!! :D *DOIN THE HAPPY DANCE*
I know the feeling, I once needed 2 days for a error in a looooooong SQL-Query, once I found the fault... then I was jumping for joy
glad you solved it
-
Re: Unknown Database Format
Quote:
Originally Posted by
ChrisE
I know the feeling, I once needed 2 days for a error in a looooooong SQL-Query, once I found the fault... then I was jumping for joy
glad you solved it
Thanks Chris! Seeing the data was easy but deciphering the *.D files header information was the main issue. Since the header varies in length between files its impossible to know where the first record starts. So figuring the *.S file was key. However once knowing how its all setup its not very difficult and is quite straight forward when you think in legacy terms.
-
Re: Unknown Database Format
Quote:
Originally Posted by
ChrisE
I know the feeling, I once needed 2 days for a error in a looooooong SQL-Query, once I found the fault... then I was jumping for joy
glad you solved it
Thanks Chris! Seeing the data was easy but deciphering the *.D files header information was the main issue. Since the header varies in length between files its impossible to know where the first record starts. So figuring the *.S file was key. However once knowing how its all setup its not very difficult and is quite straight forward when you think in legacy terms.