[RESOLVED] Proportioning Conundrum

[Shaggy Hiker]

I'm going to ask a question that makes little sense, so I will try to explain it thoroughly. This is a real world example, not something contrived, despite what it might sound like.

I have a fish trailer. I bring in two groups of fish (A and B). I don't know exactly how many fish I bring into the trailer each time I bring fish in, but I do know exactly how many fish I take out of the trailer. The sum of the fish brought in, which is unknown, is equal to the sum of the fish taken out, which is known precisely.

These two groups need to be kept distinct. Therefore, if I bring in an estimated 5,000 A and 5,000 B, the ratio of A:B is 1:1, as far as I can tell, which is good enough. Therefore, if I take 9k fish out of the trailer, leaving a bunch, the number of A removed from the trailer is estimated to be 4,500 and the same for B. This is straight proportioning. If my original estimate of the number of fish brought into the trailer is correct, then there would be 1000 fish remaining in the trailer, of which 500 are A and 500 are B.

I then bring another 9,500 B fish into the trailer, such that I estimate that there are 500 A and 10,000 B. A now makes up 5% of the total.

That would work fine. The problem is that if my initial estimate of the number of fish brought into the trailer was wrong, and there were actually 12,000 fish, then after removing the 9,000, there would be 1,500 A fish remaining in the trailer. The addition of the 10,000 B fish would result in a total in the trailer of 13,000, and the A fish comprise 12% of the total.

What I am trying to do is work out a system that would allow me to figure out how many fish were of each type when a group is taken out of the trailer. Simple proportioning if the numbers coming in are known precisely, but not so simple if they are not known.

One thing that I have worked out is that the numbers coming in will be 'rectified' once the fish trailer is emptied. At that point, the total number that were brought into the trailer is known, and all the various inputs can be adjusted to be more accurate, and as good as I can get them. At that point, normal proportioning math can work.

One option would be to say "some A and some B were moved out", and fix the numbers once the fish trailer empties. That's a convoluted solution, but it would work well. I see no other viable alternative, so I thought I'd post this here to see if anyone else has any suggestions.

Keep in mind that 10k could be estimated to have been brought in, then 10k could be counted as having been taken out, yet there could be a couple thousand fish left in the trailer. The user will note that fish remain, but they will not be expected to estimate how many (which would be so utterly error prone as to be meaningless). Therefore, until the trailer actually empties, there can easily be more fish taken out of the trailer than the number that were brought in.

Any thoughts?

[jemidiah]

What do you mean by

One option would be to say "some A and some B were moved out", and fix the numbers once the fish trailer empties.

The information you have (a guess of how many fish get put in, with a high error margin) just isn't enough to constrain the proportions very well. You could work out exactly how error prone the final proportions are with statistics and a few more assumptions (namely the accuracy of the initial estimate), but it's tedious and unhelpful since you know the answer is "too error prone". So, you need to be looking for more sources of information. Here's a few:

Find a better indirect estimate of the number of fish left after 9000 have been taken out. For example, using your numbers, instead of dumping in ~10000 B fish, dump in 5000 B's. Catch 200, estimate the fraction of A:B using that sample, and dump in more B's as needed. (Tweak "5000" and "200" as needed. The "200" could be improved with some statistics without much trouble. Add more than one round if needed.)

Find a better direct estimate of the number of fish left after 9000 have been taken out. For instance, do 1500 fish eat a bucket of food in 1/3rd the time that 500 fish do? [I don't know much about fish so I don't know if there's any sane, reliable estimator here, but I figured I'd include it.]

Find a way around needing an estimate of the number left after 9000 have been taken out. As you mentioned, if you can wait till the trailer empties, you're fine. Can you simply empty it every time instead of leaving some in?

Reduce the error margin in your estimates. For instance, figure out how error-prone your estimates really are, say, by guessing how many you take out and checking the actual number, many times. Are they consistently high by 1000? Can you get good enough at it with practice to use your estimates?

[Shaggy Hiker]

Well, that's the answer I feared I would get, and have already laid out the design to get around it, which can only be done by making a rough guess at first, which allows me to write the actual data rows needed, even though the numbers will not be correct, then correcting the numbers when the trailer empties. From a coding perspective, it's a burden, but in reality, it will probably have little impact, since the trailer will empty every day, so the delay between getting a false initial number, and fixing the number, will never be more than about 12 hours, which is too quick for any problems to arise.

I did leave out one joyous bit of information: There is no way short of genetics to determine whether a fish is an A or a B, and genetics would be prohibitively expensive in both time and money, so there is no way to tell whether a fish is an A or a B.

Oddly, the scenario I am trying to solve should NEVER happen. It's nothing but an edge case, but one that I feel the code has to be able to handle correctly because NEVER has an unusually high probability of occurence when it comes to fish.

Unless anybody else has any further commentaries, I think I shall resolve this.