I want to tokenise a string using shell delimiters...

|, >, <, >>, <<

I can do that fine, except when the string is like:

command1>command2

(i.e - no spaces between commands and tokens)

Any ideas?