How does your company deal with massive email accounts?

[stanav]

Hi everyone,

My company email server is a Linux box running @Mail software. To protect users emails in case their computer HDD's crash, we set up email accounts using IMAP and the server is backed up each night. This approach works somewhat for people with little emails (say < 1GB of emails in their Inbox), but many users have the habit of keeping everything, even spam. And over the years, their mailbox keeps being up sized whenever it gets full (instead of asking them to delete their emails). Some of the users now have more than 50GB of emails... Most users run somewhere from 5GB to 15GB... This massive amount of emails causes their email client (Outlook) to slow down alot especially when they first open it (due to all the synchronization going on between the client and the server).
So my question is, what is the email solution your company have to deal with massive amount of emails?
All inputs are appreciated.

Stanav.

[techgnome]

For starters, large servers.
Here's it's Exchange and Outlook all the way around, so there are no local pst files on the local machine. I know there's limits on mailbox size and if you start getting close, warnings go out, which if you ignore, you'll find your older stuff being purged. I don't know what the other thousands of users do, but in my case at least, I set up my email for auto archiving, and it goes into a pst file in my user folder on the network. Anything older than a month goes into that archive. But then, I'm pretty good at keeping my inbox clean in the first place.

I've just realized I wasn't any help.

-tg

[Siddharth Rout]

what is the email solution your company have to deal with massive amount of emails?

1) Emails are filtered and blocked if found inappropriate so it never reaches the user. An intimation is sent to the user that the a particular mail has been blocked and if the user feels that it is an important mail then the user has to mail the IT Dept for the same.
2) Stringent policy ensures people don't get into chain mail and stuff.
3) People have been informed to remove unnecessary mails.
4) psts are saved on the server.
5) In BPO's (where I work) people hardly stay more than 4-5 years in 1 company. So the data is automatically deleted (after careful consideration) after they leave.

[drivenbywhat]

Well the first thing you need to do is if you don't have an email company policy setting forth rules & expectations then make one asap! It's really surprising that users are allowed to have email upwards of 1gb! You need to set an inbox limit and enforce it when people reach it instead of increasing their inbox size. Just think of the bandwidth consumption on your network when they are opening their email client every morning! There's also the huge amount of data that you need to backup from the mail server. Obviously depending on your role in the company you will probably need to get management involved as well as HR to have the policy signed by each employee and put in their file. So to reiterate make sure you have a company policy that besides other things also states the size of the inbox, what happens when it's reached, automatic deletion of emails older than x days/months, and I'm sure you can think of some more. But most important don't keep letting users abuse email privileges by not doing anything about it and continue as things were!

[chris128]

All of our support clients use Exchange (well, one uses Lotus Domino/Notes but we dont talk about that...) and the mailbox policies vary from client to client. Some have mailbox limits set in Exchange so that users get a warning when their mailbox gets to a certain size (as TG mentioned) and then if it gets much larger than that (so the user has ignored the warning) then they can no longer send emails - this works fairly well but of course you do get some users saying that they NEED a larger mailbox, so some users have a slightly larger limit than others.
Some of our other clients use archiving software together with the mailbox limits mentioned above - so basically any emails over X amount of days old get compacted and held in an archive on the server. Then each Outlook client has an addin installed that lets them retrieve these archived emails if necessary. It works pretty well and is becoming more popular with quite a few of our clients going down that route now, but I dont know if there are any such system for Linux. I believe Exchange 2010 will include archiving features right out of the box, so that should be good as we will no longer need to deploy these 3rd party addins that occasionally cause issues.

[stanav]

Thanks guy for all your inputs/suggestions. We had considered switching to MS Exchange in the past, but after serious comparison for the cons and pros between our current email solution and an Exchange server, while the Exchange server has a few nicer features that our current email solution (@Mail) doesn't have (message recall, for example), it still will not solve our problem. The problem here is the mail policy that our company employed: No policy. Yes, it's a family own business. At first, the owners and the top level management people all don't want to have any kind of restrictions on their accounts. Then soon follow lower level management people in other departments claiming that they need to keep all the emails indefinitely for audit purposes. Then follow the regular employees... Until now, it's very much that every email account is warranted unlimited storage space. The initial mailbox size is set at 2GB, and every time it's filled up, the size will be doubled up. That's the policy. When a mailbox has reached its maximum storage space limit (about 16GB - limited by the software), older messages are moved to a 2nd "archive" account and keep going... So a user can have archive1, archive2, archive3... besides the primary mailbox... While this approach works, we keep getting complains on email slowness every day. And telling people to detele their old emails is not an option.
That's why I'm looking for some other approach/solution that can address our problem. I know the root of the problem is not having a firm policy on emails, but trust me, it's just isn't possible to enforce one at our organization and still have a job.

[techgnome]

Does your current system have any kind of archival methods? It may be possible to copy anything older than 6 months, put it into a secondary off-line storage, then purge the live system.

Oh... I see that's basically what you are doing. What about re-locating the archives to a different server? The only thing I can see that's going to help (and even then it may not) is to just keep expanding the server storage. You may want to put together some sort of cost-benefit analysis of keeping the "no-policy" policy vs. enacting a reasonable policy. Then if changes work best from top-down, take it to the owners. If things work better from bottom-up take it to your boss... or what ever works best.

-tg

[chris128]

I agree - discuss it with the relevant people that can do something about it. Just explain the situation to them and if they say they dont care if its getting slower they want to keep the larger mailboxes then you can tell that to your users when they complain. At the end of the day its not your fault that a) the users have massive mailboxes, and b) the email software gets slower the more data that it has to deal with. So dont be afraid to tell people that if they want to keep massive mailboxes then they will have to either put up with it being slow or buy a faster server (or upgrade the current one), just in a polite way of course

TG - expanding the storage size on the server will not make the server any faster, unless of course it is currently completely full to the point where it cannot operate normally - but Stanav hasnt mentioned any disk space issues so I dont think that is a problem at the moment.

PS message recall in exchange isnt that great... there are certainly much better reasons to move to it for starters, this current problem you have now could be addressed by using 3rd party archiving software that I mentioned (assuming nothing like that exists for this linux mail system).

[techgnome]

TG - expanding the storage size on the server will not make the server any faster, unless of course it is currently completely full to the point where it cannot operate normally - but Stanav hasnt mentioned any disk space issues so I dont think that is a problem at the moment.

No, it won't make things faster... but unless the data is archived and moved off server, expanding the server capacity is going to be the only option as the current system continues to grow.

I'm not convinced a faster server is the answer either... it just gives the illusion that it's been "fixed" ... how much longer will it be before even the faster server has issues? And then you're having this conversation all over again.

Personally I'd be ruthless about it, I'd take anyone's archive over a year old, move them into a new location, with appropriate naming so you know which archive belong to whom, compress them and store them off line. Then drop the archive from the user's view. - the kicker: Don't tell them... you might be amazed at how many probably wouldn't even notice.- They get the last year's worth of emails. If they need more than that for what ever reason, then you can either burn their archives to disc and let them search through that or temporarily restore the archive for them, and then take it back off line when they are done.

But I suppose it would depend on how much lattitude you're given in maintaining this stuff.

-tg

[chris128]

I'm not convinced a faster server is the answer either... it just gives the illusion that it's been "fixed" ... how much longer will it be before even the faster server has issues? And then you're having this conversation all over again.

Yeah sorry I didnt mean that would be a fix, I just meant when you explain these things to the people in charge you have to give them the alternatives - in this case the only way to stop the email system being slow is to either buy a more powerful server or get people using smaller mailboxes. When faced with either having to spend money or having small mailboxes, the majority of business owners / bosses will decide that having smaller mailboxes for normal users is the way to go... unless of course they have plenty of cash to spare and dont mind spending it on an upgraded server, in which case then I dont see any reason not to just get a beast of a server, as long as you warn them that somewhere down the line they will be in the same situation. One thing that IT people often tend to forget is that IT is only there to help the business run and make money - a lot of IT departments I have worked in act as if they own the network and everyone must obey their rules even if it means a user cant do their job as efficiently as they could if they didnt have to obey some of these rules (of course some rules cant be broken for good reasons, but I mean situations where IT just say something cant be done because it means more work for them or because of some policy they have just dreamt up - I'm not suggesting that this is such a case by the way Stanav!).