Printable View
Hello ,
This is a real time data.
I data will be inflowing continously. I have to differntiate the wave which have sudden change (rise or fall). and the waves(data) having gradual change(rise of fall). Consider data is height of wave with normal ups and down. if there is big change i have rise a flag.
i tried with taking 4th order differnce and compare with previous 200 values standard deviation.
But trying to findout more reliable algorithm.
thanks in advance.
Cheers.
can you post some sample data...
Thanks for u r response. I have attached two files one is excel with the data and other is bmp with the graph related to the data and a small description
The data is height of wave taken at every second.
Thanks in advance.
Cheers.
Below is an image showing your data, the top is as is, the bottom shows the moving average over 20 readings. I monitored the last 20 moving average values to find the highest and lowest value and drew a vertical line when the difference was over 300... it seems to work okay. Is a twenty second delay acceptable?
Hello,
Thanks a lot.
20 second delay is ok.( i think so).
"I monitored the last 20 moving average values to find the highest and lowest value "
Can you explain me what is that highest and lowest value.
I think, you have taken the previous 20 values and findout the average , let it be prevAvg.
then take the next 20 values and findout average, let it be nextAvg.
if(absolute(prevAvg - nextAvg) > 300)
please make comments on my understanding of your logic.
Thanks once again
Not quite, value 1 was the average of readings 1 to 20, value 2 is the average of 2 to 21, value 3; 3 to 22... etc, until I have 20 new values. Then iterate though these 20 new values, find the highest and the lowest value and compare the difference. This is done for every new value.
Can you compile VB6.0? I'll post the project if you want.
hello,
please send post me the code.
i can compile vb6.0.
I try to analyse it and may come back with few more questions.
with regards.
I've put in three scroll bars...
1 Moving average sample size
2 High/Low of averages sample size
2 The Flag point
As the moving average sample decreases it becomes less reliable, but quicker to respond.
Hello,
Thanks a lot . I will just go through it .
May come back with questions.
with regards,
Regarding your post #5, I think your assumption about what I did might well work just as well... I was concerned that taking averages in blocks would give different absolute differences depending on the offset of the block, in certain rare cases this could hide a steep height change. I had a look at this and whilst partly true it is still possible to pinpoint the big drop in the sample data you posted regardless of the offset.
It's been a long time since I studied stats.
Hello,
Now i have attached one more set of data. Please go through it once.
This time data have gradual increase, and hence the program should not
detect (turn on any flag).
It is ok Even if the time lag increases .
I wrote my own program in vc#(refering your VB code). In my program , even
in this new set of data, the flag is turning on.
Regarding your program.
why you have changed the data type to binary file. Is there any
advantage?
Thanks in advance.
It works with a moving average sample of 20, and an increase in the flagdifference to 450. As far as I can tell, the spike is over a period of 13 minutes... thats quite a long time before the data re-levels.
Edit: I'm using binary because it's smaller and I can parse it quicker. Considering a very large file, I can also move to a fixed time segment without having to count line breaks.
How much data do you have? Is it worth studying a very large sample to find the algorithm that works best? How about a multi-flag system... early warning of a potential height shift which is later confirmed or disregarded as more data comes in?
As I have already told, data is real time data .I recieve data for every second.Quote:
How much data do you have? Is it worth studying a very large sample to find the algorithm that works best? How about a multi-flag system... early warning of a potential height shift which is later confirmed or disregarded as more data comes in?
what is multi-flag system. can you brief it or suggest me some site.
Thank you.
[ sorry, we have holiday so late reply] .
If it is real time data then you should have quite a lot to study, you have posted two samples of roughly two hours.... how about 100 hours worth. The more prerecorded data to study, the better the live data algorithm could be.
By multi flag I mean... 1st flag at 10 seconds, It might be a height change. 2nd flag at ten minutes, it's defiantly a height change.
Hello,
what is live data algorithm.
By the way, can u post me the program how you convert my txt or xls files into binary .bin files.
Thanks a lot.