So I have the old db system (accessed with a terminal emulator) that I need to export the data out of but dont know what the back-end is using. Its comprised of 3 files per table it appears. a *.S, *.I, *.D
Figure maybe someone tried to get job security by renaming the extensions perhaps. opening them with a hex editor it looks like the S is a structure file (table definition), the I file possibly an index file and the D file for the data.
Any reputable website recommendations where I could upload the files to for analysis would be great. Tried a few and always fail to come up with an answer.
Thanks
VB/Office Guru™ (AKA: Gangsta Yoda™ ®)
I dont answer coding questions via PM. Please post a thread in the appropriate forum.
Could you post the first 100 bytes (or how many ever) of each file?
Last edited by Zvoni; Tomorrow at 31:69 PM.
----------------------------------------------------------------------------------------
One System to rule them all, One Code to find them,
One IDE to bring them all, and to the Framework bind them,
in the Land of Redmond, where the Windows lie
---------------------------------------------------------------------------------
People call me crazy because i'm jumping out of perfectly fine airplanes.
---------------------------------------------------------------------------------
Code is like a joke: If you have to explain it, it's bad
By viewing the file properties I get "Preprocessed C/C++ Source (.I)"
By going online using this tool: https://www.aconvert.com/
I get (drum roll) .TRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRR.......
"Congratulations! You found a file that isn't one of the 8000+ file formats that we can recognize."
By viewing the file properties I get "Preprocessed C/C++ Source (.I)"
By going online using this tool: https://www.aconvert.com/
I get (drum roll) .TRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRR.......
"Congratulations! You found a file that isn't one of the 8000+ file formats that we can recognize."
Yea the I extension shows in Windows as a C/C++ file but I feel the extension has been renamed for its original database file extension.
VB/Office Guru™ (AKA: Gangsta Yoda™ ®)
I dont answer coding questions via PM. Please post a thread in the appropriate forum.
I've looked at the files, and can't make heads or tails of it.
As for renaming the file-extension: How is the client supposed to be working, if the client looks for, say "04.idx", but it's renamed to "04.i"?
I don't even see a big difference between the D and I file. Yes, the S-File is pretty short and compact.
To me, the D-File looks like a ressource-file for a Menu in a UI. Just open it in pure notepad, and you'll see what i mean
Last edited by Zvoni; Tomorrow at 31:69 PM.
----------------------------------------------------------------------------------------
One System to rule them all, One Code to find them,
One IDE to bring them all, and to the Framework bind them,
in the Land of Redmond, where the Windows lie
---------------------------------------------------------------------------------
People call me crazy because i'm jumping out of perfectly fine airplanes.
---------------------------------------------------------------------------------
Code is like a joke: If you have to explain it, it's bad
Yes I first opened the files in a hex editor and notepad++ but figured nothing more than the S is the possible table definition. Its a very old system from the 80s at least.
I'll upload some larger files in a few if I can that dont have sensitive info in it if I can find them.
VB/Office Guru™ (AKA: Gangsta Yoda™ ®)
I dont answer coding questions via PM. Please post a thread in the appropriate forum.
The files (at first) look like old DBase (.dbf) files, but after checking in hex viewer and the dbf files structure reference, they are not. Other memories I have are about old Paradox (made by Borland) but quick searches didn't helped me to find file structure and compare how bytes "fit".
to hunt a species to extinction is not logical !
since 2010 the number of Tigers are rising again in 2016 - 3900 were counted. with Baby Callas it's 3901, my wife and I had 2-3 months the privilege of raising a Baby Tiger.
Just tested with two Paradox viewers for .db files (first renamed files to a/b/c.db) and no luck - all are invalid according to the viewers. One of the apps is using Borland database engine so it ensures that these are not Paradox files.
to hunt a species to extinction is not logical !
since 2010 the number of Tigers are rising again in 2016 - 3900 were counted. with Baby Callas it's 3901, my wife and I had 2-3 months the privilege of raising a Baby Tiger.
Seems the D and I files are binary and the S file is text
pitty, well you know now it's not a Borland File Format, I'll ask someone I know,
he's retired by now I think, I'll let you now if I have any luck
to hunt a species to extinction is not logical !
since 2010 the number of Tigers are rising again in 2016 - 3900 were counted. with Baby Callas it's 3901, my wife and I had 2-3 months the privilege of raising a Baby Tiger.
After looking at the .main executable file in a hexeditor it appears that it contains all the fields which may just be class properties BUT also found "billing_billto.idx" which I assume is an index specifier on that billing table
VB/Office Guru™ (AKA: Gangsta Yoda™ ®)
I dont answer coding questions via PM. Please post a thread in the appropriate forum.
After looking at the .main executable file in a hexeditor it appears that it contains all the fields which may just be class properties BUT also found "billing_billto.idx" which I assume is an index specifier on that billing table
that's helpfull -> .idx
was used buy
- dBase
- Visual Foxpro
- Informix
to hunt a species to extinction is not logical !
since 2010 the number of Tigers are rising again in 2016 - 3900 were counted. with Baby Callas it's 3901, my wife and I had 2-3 months the privilege of raising a Baby Tiger.
you can also google for elevatesoft they have "Additional Software and Utilities"
I think it's called dbsys.exe for opening the Files
to hunt a species to extinction is not logical !
since 2010 the number of Tigers are rising again in 2016 - 3900 were counted. with Baby Callas it's 3901, my wife and I had 2-3 months the privilege of raising a Baby Tiger.
RoboDog, the links are explaining about encryption mostly, but the files you provided in hex/text view are looking that they are not encrypted:
If someone is interested, this view is from Far Manager, a clone of the (very old) Norton Commander, but now it is 64 bit Windows app, which is updated nowadays and is open sourced.
Edited:
Some questions:
- So the db is running on Linux, created in 80s (pre-Linux era), there are Windows clients?
- How data is accessed: via client-server connection like SQL queries or another API?
- Old way (as I have seen during years) to access database files is to grant network shares (for Windows apps) that can access these database files. Is it something similar (as opposite to SQL-like or another API queries)?
Hope these questions help getting more info on the problem.
Last edited by peterst; Jul 11th, 2019 at 01:06 PM.
Reason: Added more info
RoboDog, the links are explaining about encryption mostly, but the files you provided in hex/text view are looking that they are not encrypted:
If someone is interested, this view is from Far Manager, a clone of the (very old) Norton Commander, but now it is 64 bit Windows app, which is updated nowadays and is open sourced.
Edited:
Some questions:
- So the db is running on Linux, created in 80s (pre-Linux era), there are Windows clients?
- How data is accessed: via client-server connection like SQL queries or another API?
- Old way (as I have seen during years) to access database files is to grant network shares (for Windows apps) that can access these database files. Is it something similar (as opposite to SQL-like or another API queries)?
Hope these questions help getting more info on the problem.
Its not encrypted as I have been able to read most of the data and determined which are teh Structure, Index and Data files but am unsure if they camouflaged it was my main point in posting the link..
Yes its running on a Linux server (since the early 00's was when the server was upgraded from Unix I believe).
The data is accessed via a terminal emulator session from various windows clients
No file sharing AFAIK
Appreciate teh help! Im going to look at Far Manager now, Thanks
VB/Office Guru™ (AKA: Gangsta Yoda™ ®)
I dont answer coding questions via PM. Please post a thread in the appropriate forum.
Can you show some examples how it is used from the terminal emulator? You can hide sensitive information by replacing with something irrelevant to the queries and result or just mask it in Paint or whatever you use,
It is hard to tell from the terminal window screen capture what software might be running. Is there any chance of getting a file / directory list of what is actually on the server's file system for that database? That might give us a clue.
We created our own database formats in the 70's and 80's - actually into the early 1990's for Digital Equipment PDP's and VAX's.
Our own INDEX files, our own sort-pointer files. Our own schema files and database formats. And all of it ran in the functions linked directly into the executables - no server software to find that might help you figure out what the database was.
Can you change a field or two in that screen - save the record - and then look for differences in the file after that change. That will point you in the direction of where the data lives. Might be a good first step to take...
Can you find any UI's that allow you to work the schema at all? Maybe look around for UTIL or UTILITY folders?
*** Read the sticky in the DB forum about how to get your question answered quickly!! ***
Please remember to rate posts! Rate any post you find helpful - even in old threads! Use the link to the left - "Rate this Post".
It is hard to tell from the terminal window screen capture what software might be running. Is there any chance of getting a file / directory list of what is actually on the server's file system for that database? That might give us a clue.
Its not really that kind of a setup. Its a custom linux *.main executable accessed via any terminal emulator.
VB/Office Guru™ (AKA: Gangsta Yoda™ ®)
I dont answer coding questions via PM. Please post a thread in the appropriate forum.
We created our own database formats in the 70's and 80's - actually into the early 1990's for Digital Equipment PDP's and VAX's.
Our own INDEX files, our own sort-pointer files. Our own schema files and database formats. And all of it ran in the functions linked directly into the executables - no server software to find that might help you figure out what the database was.
Can you change a field or two in that screen - save the record - and then look for differences in the file after that change. That will point you in the direction of where the data lives. Might be a good first step to take...
Can you find any UI's that allow you to work the schema at all? Maybe look around for UTIL or UTILITY folders?
You hit the nail right on the head. As mentioned we have a linux *.main executable that when viewed in a hex editor we can see the links to each database file. The *.I appears to be *.idx though but probably just renamed.
Sent you a PM
VB/Office Guru™ (AKA: Gangsta Yoda™ ®)
I dont answer coding questions via PM. Please post a thread in the appropriate forum.
So I found a couple interesting files in the root which were not numbered like the rest. Seems they are system setup files. Dont think I can post them but heres the *.P file. I believe its mapping the data (*.D) files/tables and defines the related indexe(s) for each table. I dont see where the *.S siles relate to yet though.
See the files in post #4 where they are the menu table/file and in my screen shot 4.D is listed with "_menu_del_flag"
VB/Office Guru™ (AKA: Gangsta Yoda™ ®)
I dont answer coding questions via PM. Please post a thread in the appropriate forum.
I'm sorry that I really can't help identifying the database, but I can explain what I know, I've seen and I think:
- Some of our customers were using very old CAD system developed in early 80s and running on Unix. CAD systems usually generate a lot of data which is used by other systems like ERP (in this case MRP - mechanical resource planning) and data was written in databases (files representing tables very similar to what DBase did at that time). I haven't seen these databases but I have seen exports to text files which we imported into our system many times and they are just simple human readable form of the binary data (e.g. protobuf's compared to json's)
- According to all the discussion above, files, screenshots, etc., during these years (80s and early 90s) many companies were developing own file formats not compatible with anything else, including database files
- If the files are having undocumented format, it is still possible to make simple reader that could read them - the file structure doesn't seem too complex and the fields information is already known (it is visible in the Unix/Linux app so each field can be evaluated what type it is instead of digging into the file binary structure)
- So creating few simple routines that read the data only from single file (indexes are useless since everything will be transferred into much more useful database form), fill inside list of objects, view in form so humans can check if it is the same like in the original one. If single data file is parsed correctly, then try with other data files and fix other problems
Usually each table is stored in separate file - even nowadays many databases do that. Parse data files, skip indexes, find some relation with other file types (e.g. D to S files) by checking the UI of the old application. Field names are visible, field types types are not important because you can recognize from the app UI or just by the name, field length (important when storing in binary files) is again not so important if you can compare how the app is viewing that exact data.
This is the way I will go for such problem. Sometimes it could be very time consuming but for old systems it could be the only approach.
I have figured several things out and have concluded its a C-Tree database system and probably modified from there.
Thank You Szlamany for your help with the byte reading. I figured a faster way to read it all in and how to format is just like the hex readers do.
Still I dont need to recreate the wheel, only read in the data, indices are irrelevant as I only need the data in a read only static mode for reporting etc
Time to go to bed as have to get up for work in only a couple hours.
Last edited by RobDog888; Jul 15th, 2019 at 03:50 AM.
VB/Office Guru™ (AKA: Gangsta Yoda™ ®)
I dont answer coding questions via PM. Please post a thread in the appropriate forum.
So figured out the *.S files are the field definitions of their corresponding *.D files, fixed length records, table header size which tells me where the data actually starts. Allot of detective work but looks like I'm moving on to writing the program to read it all into SQL next. No point in spending time of the *.I index files as they are useless because we will be setting up new ones.
Thanks guys!
VB/Office Guru™ (AKA: Gangsta Yoda™ ®)
I dont answer coding questions via PM. Please post a thread in the appropriate forum.
I think I may have cracked this nut!!! *DOIN THE HAPPY DANCE*
I know the feeling, I once needed 2 days for a error in a looooooong SQL-Query, once I found the fault... then I was jumping for joy
glad you solved it
to hunt a species to extinction is not logical !
since 2010 the number of Tigers are rising again in 2016 - 3900 were counted. with Baby Callas it's 3901, my wife and I had 2-3 months the privilege of raising a Baby Tiger.
I know the feeling, I once needed 2 days for a error in a looooooong SQL-Query, once I found the fault... then I was jumping for joy
glad you solved it
Thanks Chris! Seeing the data was easy but deciphering the *.D files header information was the main issue. Since the header varies in length between files its impossible to know where the first record starts. So figuring the *.S file was key. However once knowing how its all setup its not very difficult and is quite straight forward when you think in legacy terms.
VB/Office Guru™ (AKA: Gangsta Yoda™ ®)
I dont answer coding questions via PM. Please post a thread in the appropriate forum.
I know the feeling, I once needed 2 days for a error in a looooooong SQL-Query, once I found the fault... then I was jumping for joy
glad you solved it
Thanks Chris! Seeing the data was easy but deciphering the *.D files header information was the main issue. Since the header varies in length between files its impossible to know where the first record starts. So figuring the *.S file was key. However once knowing how its all setup its not very difficult and is quite straight forward when you think in legacy terms.
VB/Office Guru™ (AKA: Gangsta Yoda™ ®)
I dont answer coding questions via PM. Please post a thread in the appropriate forum.