[RESOLVED] Unknown Database Format

So I have the old db system (accessed with a terminal emulator) that I need to export the data out of but dont know what the back-end is using. Its comprised of 3 files per table it appears. a *.S, *.I, *.D
Figure maybe someone tried to get job security by renaming the extensions perhaps. opening them with a hex editor it looks like the S is a structure file (table definition), the I file possibly an index file and the D file for the data.

Any reputable website recommendations where I could upload the files to for analysis would be great. Tried a few and always fail to come up with an answer.

Thanks

I should add that the goal here is to export the data out and into sql server.

Could you post the first 100 bytes (or how many ever) of each file?

Ok I found a set of files that shouldnt contain any sensitive info. Looks like the main menu

By viewing the file properties I get "Preprocessed C/C++ Source (.I)"
By going online using this tool: https://www.aconvert.com/
I get (drum roll) .TRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRR.......
"Congratulations! You found a file that isn't one of the 8000+ file formats that we can recognize." :lol:

Quote:

---

Originally Posted by sapator

By viewing the file properties I get "Preprocessed C/C++ Source (.I)"
By going online using this tool: https://www.aconvert.com/
I get (drum roll) .TRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRR.......
"Congratulations! You found a file that isn't one of the 8000+ file formats that we can recognize." :lol:

---

Yea the I extension shows in Windows as a C/C++ file but I feel the extension has been renamed for its original database file extension.

I've looked at the files, and can't make heads or tails of it.
As for renaming the file-extension: How is the client supposed to be working, if the client looks for, say "04.idx", but it's renamed to "04.i"?
I don't even see a big difference between the D and I file. Yes, the S-File is pretty short and compact.
To me, the D-File looks like a ressource-file for a Menu in a UI. Just open it in pure notepad, and you'll see what i mean

Yes I first opened the files in a hex editor and notepad++ but figured nothing more than the S is the possible table definition. Its a very old system from the 80s at least.

I'll upload some larger files in a few if I can that dont have sensitive info in it if I can find them.

The files (at first) look like old DBase (.dbf) files, but after checking in hex viewer and the dbf files structure reference, they are not. Other memories I have are about old Paradox (made by Borland) but quick searches didn't helped me to find file structure and compare how bytes "fit".

Hi,

the extention .i looks like the old -Intermediate file from Borland

have a look here if it's a Valid extention you'll find it here
https://www.webopedia.com/quick_ref/...nsionsfull.asp

Just tested with two Paradox viewers for .db files (first renamed files to a/b/c.db) and no luck - all are invalid according to the viewers. One of the apps is using Borland database engine so it ensures that these are not Paradox files.

The system is running on a Linux box if that helps.

Quote:

---

Originally Posted by ChrisE

Hi,

the extention .i looks like the old -Intermediate file from Borland

have a look here if it's a Valid extention you'll find it here
https://www.webopedia.com/quick_ref/...nsionsfull.asp

---

Link doesnt work and couldnt find it from the main page

Hi,

the Links works from here.

you can try and open the file with this https://filext.com/file-extension/BORLAND

if it's a BorlandFile, hope it works

Quote:

---

Originally Posted by ChrisE

Hi,

the Links works from here.

you can try and open the file with this https://filext.com/file-extension/BORLAND

if it's a BorlandFile, hope it works

---

Ok that link worked. It came back with "It's a text file, but the file format is not in our database."

Quote:

---

File name: 5003.D
File type: binary file
File size: 53 MB (1% are shown)
File date: 7/9/2019, 2:14:28 PM

The file format could not be recognized.

---

Seems the D and I files are binary and the S file is text

Quote:

---

Originally Posted by RobDog888

Seems the D and I files are binary and the S file is text

---

pitty, well you know now it's not a Borland File Format, I'll ask someone I know,
he's retired by now I think, I'll let you now if I have any luck

After looking at the .main executable file in a hexeditor it appears that it contains all the fields which may just be class properties BUT also found "billing_billto.idx" which I assume is an index specifier on that billing table

Quote:

---

Originally Posted by RobDog888

After looking at the .main executable file in a hexeditor it appears that it contains all the fields which may just be class properties BUT also found "billing_billto.idx" which I assume is an index specifier on that billing table

---

that's helpfull -> .idx
was used buy
- dBase
- Visual Foxpro
- Informix

Found "C-Tree(tm)" in the hex. Googling now

you can also google for elevatesoft they have "Additional Software and Utilities"
I think it's called dbsys.exe for opening the Files

Couldnt find the file on the site and probably have to create an account to download a trial version

I did find this which is very interesting.
https://docs.faircom.com/doc/ctreeplus/74299.htm

I think this may be more of what its set up as
https://docs.faircom.com/doc/ctpodbc/37896.htm

Perhaps I can connect to a file using an appropriate odbc driver

RoboDog, the links are explaining about encryption mostly, but the files you provided in hex/text view are looking that they are not encrypted:

https://asmlair.net/pics/vbforums/cr...b_hex_view.png

If someone is interested, this view is from Far Manager, a clone of the (very old) Norton Commander, but now it is 64 bit Windows app, which is updated nowadays and is open sourced.

Edited:
Some questions:

- So the db is running on Linux, created in 80s (pre-Linux era), there are Windows clients?
- How data is accessed: via client-server connection like SQL queries or another API?
- Old way (as I have seen during years) to access database files is to grant network shares (for Windows apps) that can access these database files. Is it something similar (as opposite to SQL-like or another API queries)?

Hope these questions help getting more info on the problem.

Quote:

---

Originally Posted by peterst

RoboDog, the links are explaining about encryption mostly, but the files you provided in hex/text view are looking that they are not encrypted:

http://www.vbforums.com/

If someone is interested, this view is from Far Manager, a clone of the (very old) Norton Commander, but now it is 64 bit Windows app, which is updated nowadays and is open sourced.

Edited:
Some questions:

- So the db is running on Linux, created in 80s (pre-Linux era), there are Windows clients?
- How data is accessed: via client-server connection like SQL queries or another API?
- Old way (as I have seen during years) to access database files is to grant network shares (for Windows apps) that can access these database files. Is it something similar (as opposite to SQL-like or another API queries)?

Hope these questions help getting more info on the problem.

---

Its not encrypted as I have been able to read most of the data and determined which are teh Structure, Index and Data files but am unsure if they camouflaged it was my main point in posting the link..

Yes its running on a Linux server (since the early 00's was when the server was upgraded from Unix I believe).

The data is accessed via a terminal emulator session from various windows clients

No file sharing AFAIK

Appreciate teh help! Im going to look at Far Manager now, Thanks

Can you show some examples how it is used from the terminal emulator? You can hide sensitive information by replacing with something irrelevant to the queries and result or just mask it in Paint or whatever you use,

Here's a shot of the Order menu page

It is hard to tell from the terminal window screen capture what software might be running. Is there any chance of getting a file / directory list of what is actually on the server's file system for that database? That might give us a clue.

We created our own database formats in the 70's and 80's - actually into the early 1990's for Digital Equipment PDP's and VAX's.

Our own INDEX files, our own sort-pointer files. Our own schema files and database formats. And all of it ran in the functions linked directly into the executables - no server software to find that might help you figure out what the database was.

Can you change a field or two in that screen - save the record - and then look for differences in the file after that change. That will point you in the direction of where the data lives. Might be a good first step to take...

Can you find any UI's that allow you to work the schema at all? Maybe look around for UTIL or UTILITY folders?

Quote:

---

Originally Posted by jdc2000

It is hard to tell from the terminal window screen capture what software might be running. Is there any chance of getting a file / directory list of what is actually on the server's file system for that database? That might give us a clue.

---

Its not really that kind of a setup. Its a custom linux *.main executable accessed via any terminal emulator.

Quote:

---

Originally Posted by szlamany

We created our own database formats in the 70's and 80's - actually into the early 1990's for Digital Equipment PDP's and VAX's.

Our own INDEX files, our own sort-pointer files. Our own schema files and database formats. And all of it ran in the functions linked directly into the executables - no server software to find that might help you figure out what the database was.

Can you change a field or two in that screen - save the record - and then look for differences in the file after that change. That will point you in the direction of where the data lives. Might be a good first step to take...

Can you find any UI's that allow you to work the schema at all? Maybe look around for UTIL or UTILITY folders?

---

You hit the nail right on the head. As mentioned we have a linux *.main executable that when viewed in a hex editor we can see the links to each database file. The *.I appears to be *.idx though but probably just renamed.

Sent you a PM

So I found a couple interesting files in the root which were not numbered like the rest. Seems they are system setup files. Dont think I can post them but heres the *.P file. I believe its mapping the data (*.D) files/tables and defines the related indexe(s) for each table. I dont see where the *.S siles relate to yet though.

See the files in post #4 where they are the menu table/file and in my screen shot 4.D is listed with "_menu_del_flag"

I'd guess that was a print out of the schema
Very helpful!

Quote:

---

Originally Posted by szlamany

I'd guess that was a print out of the schema
Very helpful!

---

Yes it it! I feel Im on the right track but I need a solid finding by Monday lol

I sent you an email a few mins ago btw

I'm sorry that I really can't help identifying the database, but I can explain what I know, I've seen and I think:

- Some of our customers were using very old CAD system developed in early 80s and running on Unix. CAD systems usually generate a lot of data which is used by other systems like ERP (in this case MRP - mechanical resource planning) and data was written in databases (files representing tables very similar to what DBase did at that time). I haven't seen these databases but I have seen exports to text files which we imported into our system many times and they are just simple human readable form of the binary data (e.g. protobuf's compared to json's)

- According to all the discussion above, files, screenshots, etc., during these years (80s and early 90s) many companies were developing own file formats not compatible with anything else, including database files

- If the files are having undocumented format, it is still possible to make simple reader that could read them - the file structure doesn't seem too complex and the fields information is already known (it is visible in the Unix/Linux app so each field can be evaluated what type it is instead of digging into the file binary structure)

- So creating few simple routines that read the data only from single file (indexes are useless since everything will be transferred into much more useful database form), fill inside list of objects, view in form so humans can check if it is the same like in the original one. If single data file is parsed correctly, then try with other data files and fix other problems

Usually each table is stored in separate file - even nowadays many databases do that. Parse data files, skip indexes, find some relation with other file types (e.g. D to S files) by checking the UI of the old application. Field names are visible, field types types are not important because you can recognize from the app UI or just by the name, field length (important when storing in binary files) is again not so important if you can compare how the app is viewing that exact data.

This is the way I will go for such problem. Sometimes it could be very time consuming but for old systems it could be the only approach.

Thank you for the reply. I think may be based upon one of the older systems but tweaked which is why its not being recognized by any interpreter

I have figured several things out and have concluded its a C-Tree database system and probably modified from there.

Thank You Szlamany for your help with the byte reading. I figured a faster way to read it all in and how to format is just like the hex readers do. :D
Still I dont need to recreate the wheel, only read in the data, indices are irrelevant as I only need the data in a read only static mode for reporting etc

Time to go to bed as have to get up for work in only a couple hours.


http://www.robertromo.com/ParameterFileSetupCropped.png

I think I may have cracked this nut!!! :D *DOIN THE HAPPY DANCE*

This is the "Court" table "D" file.

http://www.robertromo.com/5037.D.DecodedCropped.png

So figured out the *.S files are the field definitions of their corresponding *.D files, fixed length records, table header size which tells me where the data actually starts. Allot of detective work but looks like I'm moving on to writing the program to read it all into SQL next. No point in spending time of the *.I index files as they are useless because we will be setting up new ones.

Thanks guys!

Quote:

---

Originally Posted by RobDog888

I think I may have cracked this nut!!! :D *DOIN THE HAPPY DANCE*

---

I know the feeling, I once needed 2 days for a error in a looooooong SQL-Query, once I found the fault... then I was jumping for joy

glad you solved it

Quote:

---

Originally Posted by ChrisE

I know the feeling, I once needed 2 days for a error in a looooooong SQL-Query, once I found the fault... then I was jumping for joy

glad you solved it

---

Thanks Chris! Seeing the data was easy but deciphering the *.D files header information was the main issue. Since the header varies in length between files its impossible to know where the first record starts. So figuring the *.S file was key. However once knowing how its all setup its not very difficult and is quite straight forward when you think in legacy terms.

Quote:

---

Originally Posted by ChrisE

I know the feeling, I once needed 2 days for a error in a looooooong SQL-Query, once I found the fault... then I was jumping for joy

glad you solved it

---

Thanks Chris! Seeing the data was easy but deciphering the *.D files header information was the main issue. Since the header varies in length between files its impossible to know where the first record starts. So figuring the *.S file was key. However once knowing how its all setup its not very difficult and is quite straight forward when you think in legacy terms.

Quote:

---

Originally Posted by RobDog888

Thanks Chris! Seeing the data was easy but deciphering the *.D files header information was the main issue. Since the header varies in length between files its impossible to know where the first record starts. So figuring the *.S file was key. However once knowing how its all setup its not very difficult and is quite straight forward when you think in legacy terms.

---

I am curious, was this program still in action ? or are you just saving/rescuing old Data for a Company

Quote:

---

Originally Posted by ChrisE

I am curious, was this program still in action ? or are you just saving/rescuing old Data for a Company

---

Its still in everyday action! We have a need to be off of it in 2 months. Should be no problem now.

I'm amazed it is still in action, because of the age, it is probably only 16bit or less. I wish my clients would of let me use a emulator, I could of keep programming in Clipper till the day I retired.

Glad you solved your problem, tough problems can cause a lot of stress but when you solve them, it's a great feeling of accomplishment..

that is amazing, that last old system in use I encounterd was 2006
there was nothing wrong with it, the CEO just wanted it ..New..
it was for Logistic