File Format Design

In order to improve my skills I decided to embark on a project for which I
will be designing a new file format.

I have done quite a bit of searching, but the only two links I was able to find
which dealt with general File Format design are

http://www.magicdb.org/filedesign.html
and
http://decoy.iki.fi/texts/filefd/filefd

I was wondering if any of you have any suggestions, recommendations,
warnings, annecdotes you could share.

I would also be grateful if you could advise whether it is better to use
Big-Endian encoding or Little-Endian encoding.

It really doesn't matter that much or require too much thought unless you're looking to standardize your file format. Typically, you want to write your data as dense as possible; unless you want to go for human readable in which case, I'd just use XML.

Since x86 CPUs use little-endian I'd keep it at that. I don't think there are many CPUs left that use big-endian.

The first thing you should be asking yourself is WHAT do you suppose to store in your files.
Depending on the task you should design its features.
Take ISO, DBF and AVI or TXT - they all organized differently because they were designed for different tasks and to store different things. As soon as you decide what exactly your new file format is intended for the picture will be clearer.