Tags: 10-100, analyzes, application, arrays, cluster, functional, global, labview, program, programming, quotbuildingquot, returns, running
"Building" large arrays using functional global
I have an application which analyzes data and returns a "results" cluster at
10-100 Hz. This program could be running from minutes to days a
nd I'd like to have access to all of the results clusters meaning if I store
them in an array it could
have something on the order of a million elements. I've written a func
tional global subvi that preallocates an array for the results and resizes i
t when the array becomes filled (doubling the size each time). This av
oids both having to allocat
e a huge array to begin with and using the "build array" vi too often.
This works but still doesn't seem that fast to me. In my actual appli
cation I have clusters with on the order of 100 elements and even when my ar
ray is only a couple thousa
nd elements big the profiler is showing that this vi is taking a few millise
conds each time. I've attached a sample version of the subvi and its b
lock diagram below. Are there things I can do to make this faster?&nbs
p; Thanks,Paul<img src="/app/links/?src=htt
29_bd0.jpg"> Message Edited by Phamton on 01-08-2008 01:36 PM
Resizing array (cluster).vi:
http://forums.ni.com/attachments/ni...3644/1/Resizing array (cluster).vi
Resizing array (cluster)_bd0.jpg:
http://forums.ni.com/attachments/ni...3644/2/Resizing array (cluster)_bd
Leave a comment...
- 5 Comments
- Hello Paul, one easy mod is to physically move all constants in your diagram
that are inside the while loop to outside the while loop, and of course con
nect them back to necessary inputs inside the loop. I think that's a speed u
p hint I recall from past d
Good Luck - Brian#1; Thu, 01 May 2008 01:42:00 GMT
- First of all, nice work on the functional global.You might try separating yo
ur Get and Set operations. Currently, everytime you Set an item into storage
, you also Get a copy of the whole storage. I'd guess it's this extra copy t
hat's hurting performance.I
f you are setting multiple packets, but only getting the results for analysi
s every once in a while, this should have a big effect on performance. Check
out the two pics below. Notice that in the Set case, you're only outputting
an empty array, and not ma
king a complete copy of your internal data.tachments/ni/170/293653/1/Get.PNG"> ts/ni/170/293653/2/Set.PNG"> Message Edited by Jarrod S. on 01-08-2008 02:0
Set.PNG:#2; Thu, 01 May 2008 01:43:00 GMT
- Couple of thoughts...
When I last analyzed the time involved in building an array (LV 5.1?), I can
to the conclusion that LV started out with about 1K by default. As long as
the buffer stayed below 1K, I did not see delays trying to build. If I conti
nued to build I saw another
hit at 2K, 4K, etc. So... I think all of your work is just duplicating what
the build array in a loop does.
Another approach (remember this is coming from Ben, Mr AE) is just use multi
ple queues. One for the GUI, one for logging, one for the analysis. The GUI
and Logging queues you keep reading as normal. THe analysis queue just gets
read once at analysis time.
How about using the in-place operations. They let you work exclusively in-pl
Yet another thought...
How about pre-allocating 5X what you need? THen just take the sub-set when i
t comes time to read.
Queues will often out perform AE since they can work in-place.
Ben#3; Thu, 01 May 2008 01:44:00 GMT
- "...from minutes to seconds and my code is way simpler;..."
See <a href="/app/links/?link=http://forums.ni.com/ni/board/message?board.id=170&view=by_
date_ascending&message.id=265705#M265705" target="_blank">here</a> for i
So are we done here?
Ben#4; Thu, 01 May 2008 01:45:00 GMT
- Phamton wrote:
I hadn't noticed the nifty option of "Return elements?" in the Get Queue Sta
Yes, I always forget about that as well, or I would have mentioned it. :smil
from minutes to seconds
I have to admit I didn't expect such an improvement when I originally sugges
ted this, because your global was already relatively good, but I guess it de
pends on the specific implementation.
Note that queues can operate in-place, but they might still create copies, d
epending on what you do.#5; Thu, 01 May 2008 01:46:00 GMT