How to identify an original file from copies

[Witis]

Hi

To start with I am trying to work out how to differentiate an original file from a CD copy. I thought that I might be able to use the "date created" file attribute.

As a result I am trying to work out what happens to the "date created" (not date modified) attached to a file when it is written/burnt to a cd or dvd (right click the column headers in file explorer and check "date created" to view the value). Is the "date created' on files written to the CD/DVD supposed to be the same, always different, or different although somehow related to the "date created" on the original file?

I checked a single text file, and recorded the values before and after writing the file to a CD, and these were the results:
C:\Temp\check.txt 11/28/2011 9:26:23 PM (from HDD before writing/burning)
D:\check.txt 11/29/2011 8:26:23 AM (from the file written to the DVD drive)

To me it seems as though the values are somehow related as the minutes and seconds are identical even though the day and hours are different, although I am not sure what the relationship is. I double checked the results by erasing the disc and performing the check a second time, and got the same results.

Is this the expected result?

[Hack]

Moved To General Developer

[dilettante]

I suspect a great deal depends on the software and its options selected when doing the "burning."

Some may let you control the dates used while others may copy some or all of the timestamps from the source files.

[Witis]

Thanks for the reply Dilettante, I use burnaware, which has a free edition available (for personal non commercial use), as it allows me to use CDs just like a HDD, i.e. add and delete files and update files on the go rather than having to delete the whole disc and start over every time, and also allows for secure deleting of information from discs (rare).

It doesn't appear to have any options directly relating to the "date created" on files and I was not able to see anything in the ISO standard regarding the "date created" (http://en.wikipedia.org/wiki/ISO_9660), although there does from my checking appear to be some sort of relationship between the date created on the burnt copy from the original when using burnaware, presumably to make it easy for user to identify their burnt copies in relation to the original.

I also tried nero express 7 which I had a copy of (came with a dvdwriter), and got these results:
C:\Temp\check.txt 11/28/2011 9:26:23 PM (original)
D:\check.txt 11/28/2011 9:26:23 PM (cd copy)
Making the two files appear identical.

In theory the file is burnt to the CD at a different time to when the file was originally created, thereby creating a copy and should attract a different "date created", although there does not seem to be anything in the ISO standards which prevents burning software from using the SetFileTime api (http://msdn.microsoft.com/en-us/libr...(v=vs.85).aspx) as Nero 7 appears to have done, in order to ensure that the file stamps are identical on the copy.

Is there any other way of distinguishing a copy that has been burnt to the CD from the original?

[Nightwalker83]

Could you add a comment to the copy image specifying that it is a copy?

[Witis]

Hi Nightwalker83, one way that I think might be possible is to get the path from the filename, the drive of the path, then work out what the drive type is and therefore determine if the file is a cd copy.

[Witis]

One more thing, I am under the impression that when I copy, via ctrl + c and ctrl + v for example, only the time and date stamp changes, specifically only the "date created" is changed as in windows 7 "date accessed" is crippled by default (http://www.groovypost.com/howto/micr...der-windows-7/) and date modified is unaltered in the copy process. Is this correct or do other file properties also change when I copy files?

[Evil_Giraffe]

For further discussion on Last Access Time, see here:

http://blogs.msdn.com/b/oldnewthing/.../10222560.aspx

[dilettante]

A lot of those packet-writing schemes are proprietary and non-portable ("pretend the CD is a floppy"). There is a hardware-based standard (Mount Ranier) but non-supporting devices were so common by the time it came out that you can't count on it. Lots of the 3rd party CD support software products still insist on sing their own ad hoc formats.

Avoid using CDs/CD-RWs this way.

Microsoft tried to avoid the issue by adding a weird thing based on a hard drive scratchpad that can be burned to CDs. This can work better but takes a lot out of your control, for example control over the CD sessions. The resulting CDs can also end up being non-portable.

Doesn't help answer the question but just an FYI.


Maybe if you told us exactly what you really need to be able to do we might have a better answer. We may be trying to "debug" a dead-end.

For example if you have a program where a user tells it what file to process (via Open dialog, etc.) it is pretty easy to get the drive letter and check the drive type if you want to know if it is on a CD.

Or if you want to know whether the file is original or was transported/copied from an original via CD, Flash drive, FTP, etc... if the original was on an NTFS partition you could add a secondary stream (even an empty one). These won't be copied to most CD formats or to FAT devices, so would be lost. Your program could look for that.

[dilettante]

Here's a demo of the latter technique.

Since very few users have hard drive partitions that are FAT-formatted anymore it can be fairly widely useful.

Just another thought.

[Witis]

Hi dilettante,
Just to let you know, I am not ignoring your posts, it is just that I use operamail/fastmail which does not accept vb forum emails; from memory I think only Gmail is recommended to get around this problem although I can't see myself joining up tomorrow (http://www.smh.com.au/technology/tec...0525-w937.html).

To clarify things further, I am looking at trying to distinguish an original from a copy, and I have to say I have not seen that technique of used before, although I just checked and the validation-tag is copied when the file is copied (rather than falling off when a file is copied) and as you say it doesn't work on FAT drives.

You saw in the other thread that I was looking at using the tag/comments properties of a file or appending to the file for such a purpose, and it looks like a promising way of storing a validation tags even on FAT drives (USB). The main issue is determining what file properties actually do change when a file is copied so that I can then work out how to engineer it. So far I have:
1. File path
2. File name
3. File date created

If M$ had of kept the "date created" field away from external programmers (i.e. only implemented it internally for use by the OS) it would have made this a much easier task, even so, I have not given up yet.

[dilettante]

The alternate stream got copied to a CD? I would not have expected that at all. Maybe the software and burning style you are using does that. I was sure that "normal" CD-ROM-image ISO-9660 CDFS does not support this.

You saw in the other thread that I was looking at using the tag/comments properties of a file or appending to the file for such a purpose, and it looks like a promising way of storing a validation tags even on FAT drives (USB).

Those are part of the file's data, so they'll always be copied unchanged with any media or over FTP, HTTP, in a ZIP archive, etc.

[Witis]

I thought the text stream idea was to create a tag that fell off when it was moved to another media such as CD or FAT etc, which is one approach i.e. create a tag that falls off when copied.

Other approaches include checking the date modified and drive type to identify an original from a copy, and recording the file path and name within the exe file and only allowing it to run from one location (and then working on a way to allow the user to alter the path themselves without compromising the original).

Although it would be easier if there were more pieces to work with, which are altered when a file is copied, in addition to file name, path, and date created...

Actually I now have (only checked on windows 7 NTFS):

1. File name (different if in the same path)
2. File path (although this does not always change)
3. File date created
4. If the Read only file attribute was checked on the original, it falls off during the copy process
5. If the file is checked as "Compress contents to save disk space" on the original, it falls off during the copy process.
6. When a file is copied it is always checked as ready for archiving on the copy regardless of the original.
7. When a file is copied it is always checked as "Allow this file to have contents indexed in addition to file properties" on the copy regardless of the original.

*1 As file date last accessed is not updated by windows 7 by default, it is therefore omitted.
*2 file date modified is not altered when a file is copied and is therefore omitted.

As expected problems arise when checking on FAT drives, for a start it looks like many of the properties don't exist eg "Compress contents to save disk space" and "Allow this file to have contents indexed in addition to file properties", and in addtion:
a. The read only attribute is always set to on when transferring an original or creating a copy from an NTFS drive to a FAT drive.
b. The archive attribute is always set to on when transferring an original or creating a copy from an NTFS drive onto a FAT drive.

Which leaves the classic trio of:
1. File name (different if in the same path)
2. File path (although this does not always change)
3. File date created
when taking into account both NTFS and FAT drive formats.

[dilettante]

I thought the text stream idea was to create a tag that fell off when it was moved to another media such as CD or FAT etc, which is one approach i.e. create a tag that falls off when copied.

Yes, that's true. But very few of the extended properties you can see as Windows Explorer columns in details view (or in its Properties dialog) are implemented as alternate streams. One exception for sure is the thing you see on a downloaded file these days, that results in an "UnBlock" button in Properties and produces a warning dialog when you try to open the file via Explorer.

a. The read only attribute is always set to on when transferring an original or creating a copy from an NTFS drive to a FAT drive.

I don't think so. This would break all kinds of stuff.

Files on a CD are read-only, and those get the RO bit set when copied back to disk.

You could clear the archive bit on your "original" and it will stay off until somebody or some program turns it back on. A copy to anywhere should turn it on as far as I recall.


If this is for anti-piracy I don't think it will work. If your program expects a certain path, they can just copy the file to that path.

[dilettante]

This might be useful if you haven't seen it:

http://msdn.microsoft.com/en-us/libr...90(VS.85).aspx

And here is the "Windows 7 Washed Out Look With Crippled Navigation" .Net-age version of the same info that Google will feed you:

http://msdn.microsoft.com/en-us/libr...(v=vs.85).aspx

I encourage enveryone to scroll to the very bottom and click Site Feedback, and ask them to bring back the older legible and tree-navigable page theme to these pages.

[Witis]

Yes, that's true. But very few of the extended properties you can see as Windows Explorer columns in details view (or in its Properties dialog) are implemented as alternate streams. One exception for sure is the thing you see on a downloaded file these days, that results in an "UnBlock" button in Properties and produces a warning dialog when you try to open the file via Explorer.

Out of interest do any of the alternate textstream properties stay with files that are transferred to FAT drives, or do they always fall off this way?

I don't think so. This would break all kinds of stuff.

yup my bad.

You could clear the archive bit on your "original" and it will stay off until somebody or some program turns it back on. A copy to anywhere should turn it on as far as I recall.

Problem is the FAT format again, as soon as you try and move the original to a FAT drive the achive attribute is set to on automatically.

If this is for anti-piracy I don't think it will work. If your program expects a certain path, they can just copy the file to that path.

Yes it is for anti-piracy, I am still playing around with it and have not given up just yet, I think it might be possible even with such few levers to work with. Although I have not checked if there are any commercial options available, I am just seeing if I can get something to work for my own applications.

Another thing in relation to anti-piracy, are comments included in the compiled exe or are they discarded by the compiler?
It looks like comments are not compiled into the exe:
"Ordinarily, the VB compiler reads all the source code in a project and translates it into either P-Code or native code that can be used by a computer. It doesn’t omit a single line of executable code (naturally, comments aren’t compiled), so every instruction that was in the source code also winds up in the compiled EXE, DLL, or custom control."
source: http://visualbasic.freetutes.com/lea...son20/p19.html

[dilettante]

Out of interest do any of the alternate textstream properties stay with files that are transferred to FAT drives, or do they always fall off this way?

FAT filesystems have no alternate data streams.

Problem is the FAT format again, as soon as you try and move the original to a FAT drive the achive attribute is set to on automatically.

I thought you wanted to detect copying. That would be copying.

Another thing in relation to anti-piracy, are comments included in the compiled exe or are they discarded by the compiler?

No, comments don't get copied into the compiled program, but many symbolic names do (Forms, Classes, Methods, Properties, etc.).

[Witis]

I thought you wanted to detect copying. That would be copying.

I think that moving a file from one drive to another should be allowed, eg moving a file from a HDD to a USB drive should be allowed, I am only trying to stop multiple copies from being made. As long as there is only one original file I don't mind how many times it is moved.

I just realised that even if I can get copy protection sorted for anti piracy reasons I am not sure if it will be popular as any restriction such as only being able to move rather than copy files, even though it prevents illegal copies being made, might put some users off if they feel restricted in any way. In other words we are so familiar with being able to make backup copies of software and using one copy to install a piece of software on as many machine as we like that any reduction in these facilities might put some buyers off. It would be good to know if, on average, any reduction in popularity due to copy protection schemes is offset by an equal or larger reduction in software piracy.

[dilettante]

It really depends on your market.

If your software is mostly used by "naive" and honest users copy protection will probably just frustrate customers, so you'd want to avoid making it too onerous.

On the other hand I got out of the game market 10 years ago. A high fraction of gamers are out and out crooks and cracked software is a fact of life in their world so it just wasn't worth the effort for the low-ticket stuff I was selling.

There is very little I sell copy protected now, and I use a simple scheme. But that targets people who aren't interested in a wumpus hunt for "cracks" anyway, and the software is very cheap ($5 range).


What "real money" there is left in the market for me anymore is for custom (sometimes called bespoken) projects.

[Witis]

I am doing it the other way around, too many years working on custom apps made me compromise my work too much for the clients, so now I am trying the off the shelf approach where I get full creative control. Just about finished one project and I have another couple in development (utility products not games), so copy protection and ecommerce are new areas for me.

I wonder if I am the one being too naive if I try and sell software without any copy protection or are the majority of all regular users honest and trustworthy? I suspect that your experience with the gamers might actually translate to the wider market notwithstanding those users with lots of funds and good morals.

[dilettante]

I have one $3 program that requires a "key" to open the full feature set that gets emailed on purchase, and the key is the same for every purchase.

Works great and no serious piracy that I've noticed. Still sells at about 5 copies a month after 8 years.

[Witis]

You make it seem like software piracy is not such a large problem, what happened with your games? I sometimes work at a charity doing some network management stuff, and I have noticed cracks and hacks of just about every type of software on the market being downloaded off the net at various sites which seem to multiply all the time eg Pirate Bay, scary stuff. I usually try and guide them in the other direction, citing breaches of music copyright costing hundreds of thousands of dollars for even poverty stricken individuals, although it is hard to tell just how many people are into the illegal software downloads. The problem which seems real to me is this: just say you sell one copy of your application to one of the guys who also uses Pirate Bay for $3, he then uploads it and the key to Pirate Bay and gives it to everyone for free, and there goes your per month sales and profit.

[dilettante]

I agree, which is why you want to target markets that tend to be more honest. The kind of people who don't go looking for a "cracked version" or stolen keys posted online, let alone try to crack it themselves. You will still lose sales, but a lot of people would never have bought it anyway. I agree this is far from perfect though.

This is why software is starting to require online "activation" schemes like Windows does. Sometimes you have no choice but to put more effort into protection.

In the end I've decided not to try to sell to mass markets except in a few cases.


This is part of what is going on with Windows 8 Metro applets. Microsoft wants a piece of every sale, so they are doing everything they can to require lockdowns using code signing and doing runtime signature verification and such, and controling distribution (people can only buy from their "app store" and there will be no other way to install these Metro applets).

[Witis]

I think I can see where you are coming from: if you are making engineering, medical or legal applications then the target market is more likely to be honest and rich and not even go looking for cracks making any loss via pirate bay et al. minimal. Where as the wider market is much more likely to include dodgy types who don't mind taking a visit to the sites of lesser repute.

As far as Microsoft's current attempts to try and control piracy, it seems to me that Microsoft is a bit of a victim of their own lack of anti piracy engineering. Rather than build a operating system that protects them from piracy when they have full control over the source, they have instead chosen not to include an appropriate watermarking/anti piracy systems and are instead forced to devote lots of resources into sub optimal solutions eating massively into their profits. I don't know what the figures actually are, however, some people have suggested that 80%+ of the MS software sold on Ebay is fake and most of these are able to bypass the online activation test. Although I don't know what they are planning as far as anti piracy goes for windows 8, I only read lots of posts saying that it is going to be a bigger flop than UAC and Windows Vista due to the Metro interface.

Actually this worries me: "The Windows Store will be the only means of distributing Metro-style apps to users to allow Microsoft to scan apps for security flaws and malware.[4] The licensing and revenue share through the Windows Store have yet to be confirmed.[5] However, the Windows Store will take a 30% cut of application sales. Once that application reaches $25,000 USD in revenue, the cut drops to 20%. Individual developers will be able to register for $49 USD, and $99 USD for companies."
source:http://en.wikipedia.org/wiki/Windows_Store

Sounds like software producers will now have to pay to register to sell, then pay 30% (20% if successful) of sales to MS, and there is no other way to sell metro applications! Being forced into giving up 30% of sales revenue without any other options is abuse of market position or standover tactics and is highly offensive in my view, and it must be illegal. Surely this breaches some laws in the US for example anti competitive (antitrust) laws which usually ban abusive behavior by any firms dominating a market.

[dilettante]

It is quite similar to "app store" operations on other platforms. Microsoft is just following an established business model.


Remember, the idea with phone and tablet devices is to keep costs low and get them into the hands of people who shouldn't really be spending money on them. Then they get rich by controlling the platform and charging high usage fees with multiyear contract lock-in and sizable fees on brokered 3rd party "crapplets" designed to appeal to vanity and greed.

A tiny fraction of users manage to get their devices "jailbroken" while the rest keep pestering mom and dad for bigger and bigger allowances to blow on more crapplets and finally a new platform where it begins all over.

This is like the men's face-shaving razor market where they underprice (and even send out free) starter kits with a fancy "handle" and a few blades. Then they charge ripoff prices for additional packs of blades and control interchangability with gerenic replacements.

[Witis]

I think you mean the Apple options: Mac App Store/App Store (http://en.wikipedia.org/wiki/Mac_App_Store / http://en.wikipedia.org/wiki/App_Store_(iOS)). The earliest Apple App Store, the iOS, only opened on July 10 2008, making it a relatively recent venture. Moreover I did not read anything that suggests that all applications must go via the Apple App Stores unlike the Windows Store. If Apple is not forcing app creators to use their store then I can't oppose it, it is only when one is forced into an arrangement that takes 30% of sales, that's sales not profits, where most companies would be very lucky to get back 30% of sales as profit, that I have a problem with it. Furthermore they also hint that the code will be examined for security purposes, although I can't help think they might use the easier decompilation of .net apps to access the code and reverse engineer the more prolific or highly successful applications in order to compete against them (where as without the Window store many successful apps would avoid such scrutiny).

I absolutely agree with the shaving analogy, this Windows Store approach does in my view move MS away from a hirsute approach and towards a Gillette model with MS as top dog.

[dilettante]

I don't keep up with Apple issues so things may have changed, but:

Why Jailbreak your iPhone, iPad, iPod Touch or Apple TV?

When you first hear about the ability to jailbreak your iPhone, iPad, iPod touch or Apple TV you are probably thinking, “Why should I do this?” Well, the two main reasons are to allow programs to run that Apple doesn’t approve of and to customize the look of your device. I’m going to try and cover some of the best programs (apps) that you can run and best customizations (mods) you can do after jailbreaking your device.

Also see UPDATED: All Your Apps Are Belong to Apple: The iPhone Developer Program License Agreement


Also Apple forces Amazon to alter Kindle app:

As we reported last month, Apple's twice amended its terms for subscriptions in the App Store, which require companies to give Apple a 30 percent cut on sales their apps generate. In the past, e-reading apps Kindle, Nook, and Kobo have avoided paying the cut by sending customers to a Web-based interface outside the app.

[Witis]

Overall, the Agreement is a very one-sided contract, favoring Apple at every turn. That's not unusual where end-user license agreements are concerned (and not all the terms may ultimately be enforceable), but it's a bit of a surprise as applied to the more than 100,000 developers for the iPhone, including many large public companies. How can Apple get away with it? Because it is the sole gateway to the more than 40 million iPhones that have been sold. In other words, it's only because Apple still "owns" the customer, long after each iPhone (and soon, iPad) is sold, that it is able to push these contractual terms on the entire universe of software developers for the platform.

In short, no competition among app stores means no competition for the license terms that apply to iPhone developers.

That sounds like an antitrust issue to me, they, meaning Apple and now Windows, are trying to bully everyone into an oligopolistic nightmare, and that's when the judges and legislators are supposed to step in and say enough of these mafia like protection racket strategies designed to eliminate competition and bully the app creators into giving up all of their profits like the giant in your class at high school after your lunch money when they have no where else to go to make a living, you are simply not legally allowed to behave in such a manner and abuse your market position like that, so you have to go to jail instead of the applications and app creators.

[dilettante]

We're in a legislative and judicial environment of late that is very anti consumer and anti worker, falling over itself to be pro business. Courts have been increasingly activist in recent years to extend the scope and depth of "business rights." The bias is toward the large end, e.g. if a smaller and larger business are in a fight they'll find for the larger.

A combination of laissez-faire capitalism with a twisted addition of corporate welfare and checkbook-take-all rules the day.

[Witis]

I think you must be referring to the IT sector, and this is certainly due to an oligopolistic situation involving companies such as Apple and Microsoft.

In the relationship between the producer and the consumer, case law and legislation usually ensures that neither side takes advantage of the other. Producers can be abused by consumers if they make illegal copies (e.g. of books or software), and consumers can be stood over by monopolistic producers who seek to remove consumer rights in an attempt to gain advantages including control and financial benefits. Producers are protected by copyright laws, consumers by common law such as the doctrine of first sale and consumer legislation including antitrust laws.

At the moment the IT sector is attempting to control consumers by arguing that "The software is licensed, not sold". As licenses usually refer to contracts involving intellectual property and leases apply to physical goods, I think they probably should call it a lease rather than a license, nevertheless the idea is simple, set up a lease/license situation in order to allow the producer to exert much more control over the consumers by retaining the title to the goods they are selling. In turn this allows the software giants the ability to specify that the consumer is only allowed to on sell the software once or not at all in the case of NFR ("Not for Resale") software, or prohibit the rental of their software and other such terms which are not usually enforceable when examining a regular sale which involves the transfer of title via the doctrine of first sale.

If you have enough market power you can then choose to only offer everyone the lease/license arrangement in an attempt to force all the consumers into the terms of the lease/license. It is extreme market pressure being exerted on consumers by large multinational giants who are not subject to sufficient competition to force them to cater to consumer needs over their own.

At the same time the counterfeit market for computer software has never been more lively. So producers are trying on a technical argument in an attempt to circumvent consumer protections while at the same time breach of copyright has never been of such a high quality or more rampant where criminals view it as a chance to churn out high quality counterfeit $100-200+ bills.

Although I feel sorry for the large corporates when they have their software counterfeited, I find myself arguing against their attempts to use their market power to circumvent consumer protection and bully programmers so that they can control their products and take most of their profits.

The two possible courses of action to deal with the abuse of market position due to concentrated market power in the case of IT conglomerates such as Microsoft and Apple are as you say laissez-faire capitalism, and the other involves using legislation and the court system to actively manage the ever changing dynamics. If you do nothing, then it is possible that in the long term the situation will rectify itself as such market power indicates to everyone that there are massive opportunities and profits in that sector which should eventually create competition. Also in the current situation involving Apple and MS trying to control the smaller software companies, it might actually expedite this process, as this amount of control and profit stealing might cause these companies to start developing their own Operating Systems thereby accelerating the creation of additional players in the commercial OS market.

However, how long will it take, and how many consumers and software companies will have to suffer under these bully boy tactics until the situation resolves? That is where the courts and legislation can step in and actively manage those players who choose to abuse their success, market position and power. It is a bit like the economy, you probably don't have to manage it, however, it is much more enjoyable when it is actively managed to soften the depressions. In the case of the licensing situation some states have enacted legislation that forces all sales of software to be treated as regular sales rather than leases/licenses thereby blocking any attempt to control consumers that way. I am not sure if the App store concept has been around long enough to get any substantial opposition just yet, although it might just happen depending on the state of the antitrust laws and the depth of the pockets of some of the medium to large size companies that Apple and MS are now targeting.