ULP: Create reliable temp filename for scripts

On 06/20/2013 09:06 AM, Axel Jäger wrote:

Hello,

as far as I understand, the only way to manipulate the data structure of

either an eagle schematic, board or library through an ULP is to collect

the necessary commands in a text file and execute it using exit ("script

<filename>").

Axel,

Rather than collecting commands into a file, collect them into a string

then pass the string as the single argument to exit().

    string cmd = "";

    cmd += "display all;\n";    // display all layers

    cmd += "group all;\n";      // Group all objects

    cmd += "lock (CS> 0 0);\n"; // Unlock everything.

    cmd += "display last;\n";   // restore the original display

    ... add more useful commands to cmd

    exit(cmd);

No files are used here.

HTH,

    - Chuck

Chuck: THIS is the solution, many thanks.

Axel Jäger wrote:

Chuck: THIS is the solution, many thanks.

 

--

Chuck's solution is used often but is slow when you perform many cmd +=

Concatenating strings in a ULP is slow.

When that becomes and issue you can have your ULP create your script file in

the directory of the current  editor file you have open. You  know you can

write to that place.

You create the file using  open ("text file",wtD) so that that file is

deleted when Eagle is closed. Hence the file is temporary.

I think that covers your needs

Warren

--

Viewed / responded via the newsgroup at

news.cadsoft.de

Chuck's solution is used often but is slow when you perform many cmd +=

Concatenating strings in a ULP is slow.

Well, don't use string concatenation. Collect all the commands in a

string array and only as last step do the concatenation with strjoin().

Another benefit is you can use sprintf() and generate complex command

strings much easier. See the code below.

Kim

====================================

int ns;          //define counter

string scr[];     //define cmd array

//insert simple command/s

scr[ns++] = "set undo_log off; set confirm yes;";

//insert complex commands

sprintf(scr[ns++], "edit %s.brd;", name);

sprintf(scr[ns++], "text '%s' %s%sR%f (%fmm %fmm);", t[i].value, spin,

mirr, t[i].angle, u2mm(t[i].x), u2mm(t[i].y));

//concatenate array->string

string cmd = strjoin(scr,' ');

//exit and execute the script

exit(cmd);

====================================

Kim,

I like that even better.  I am aware of the inefficiencies of doing

multiple string concatenations.  Underneath the covers, I assume that

each concatination does a strlen() on both strings to determine the

memory required, then copies both strings to the newly allocated space.

So the cost goes up with the number and length of the strings exponentially.

strjoin(), however, looks like it can scan all the strings once to

determine memory requirements, then copying each string into the

allocated space.  So the cost is at worst 2n - more of a linear increase

in cost.  Much more CPU friendly than successive concatenations.

Thanks for pointing out a better method.  I'll put it to good use.

    - Chuck

On 06/21/2013 08:52 AM, KimP wrote:

Chuck's solution is used often but is slow when you perform many cmd +=

Concatenating strings in a ULP is slow.

 

Well, don't use string concatenation. Collect all the commands in a

string array and only as last step do the concatenation with

strjoin(). Another benefit is you can use sprintf() and generate

complex command strings much easier. See the code below.

 

Kim

 

====================================

int ns;        //define counter

string scr[];    //define cmd array

 

//insert simple command/s

scr[ns++] = "set undo_log off; set confirm yes;";

 

//insert complex commands

sprintf(scr[ns++], "edit %s.brd;", name);

sprintf(scr[ns++], "text '%s' %s%sR%f (%fmm %fmm);", t[i].value, spin,

mirr, t[i].angle, u2mm(t[i].x), u2mm(t[i].y));

 

//concatenate array->string

string cmd = strjoin(scr,' ');

 

//exit and execute the script

exit(cmd);

====================================

 

Hi

It appears there is no real speed advantage using the array/string join

method in Eagle ULP. The method is used in other coding to advantage. It is

said that many browsers have optimised their treatment of string

concatenation and have it running faster than array/string join.

I got curious to see what the advantage gain was in Eagle. Run the attached

ULP that times the two methods. The first time its run there is no

difference in time between the two methods but run it again (Go) and you

see concatenation take the same time but array/string gets quicker. I

suspect the array is still in place so the ULP processor does not need to

create it. Normally a ULP would build the exit(command string) only once so

both methods are as quick as each other.

I feel the advantage of running a script is that Eagle appears to perform

the commands from the script far faster than it processes the commands in

an exit command and advantages like speed of rendering the result in the

editor. You can see the difference on slower PCs. You can see from the ULP

that saving the text string (from the concatenation test) only adds a few

milliseconds so for me its often the preferred technique.

Enjoy

Warren

--

Web access to CadSoft support forums at www.eaglecentral.ca.  Where the CadSoft EAGLE community meets.

Interesting,

I consistently get results showing that strjoin() is about twice as fast

as the concatenate.  This is on a 3GHz Pentium running Linux 2.6.34.

But as you mentioned, it's miniscule compared to the time it takes to

execute the script.

See attached screen shot.

    - Chuck

On 06/22/2013 03:37 AM, Warren Brayshaw wrote:

Hi

 

It appears there is no real speed advantage using the array/string join

method in Eagle ULP. The method is used in other coding to advantage. It is

said that many browsers have optimised their treatment of string

concatenation and have it running faster than array/string join.

 

I got curious to see what the advantage gain was in Eagle. Run the attached

ULP that times the two methods. The first time its run there is no

difference in time between the two methods but run it again (Go) and you

see concatenation take the same time but array/string gets quicker. I

suspect the array is still in place so the ULP processor does not need to

create it. Normally a ULP would build the exit(command string) only once so

both methods are as quick as each other.

 

I feel the advantage of running a script is that Eagle appears to perform

the commands from the script far faster than it processes the commands in

an exit command and advantages like speed of rendering the result in the

editor. You can see the difference on slower PCs. You can see from the ULP

that saving the text string (from the concatenation test) only adds a few

milliseconds so for me its often the preferred technique.

 

Enjoy

 

Warren

 

 

Attachments:

Hi Chuck

My previously mentioned times were from a P500 running Win2000. I use that

PC for writing the ULP as it shows up bad coding choices. If it's fast on

that machine then its acceptable.

I timed it again using a Win7 32bit Dual Core mailto:T6670@2.20GHz

On the first pass the strJoin was 82% of the Concatenation time

Subsequent passes averaged 65% of the Concatenation time and quite

consistant.

So although strJoin took longer on the first pass it was always faster than

concatenation.

Warren

--

Web access to CadSoft support forums at www.eaglecentral.ca.  Where the CadSoft EAGLE community meets.

Chuck Huber wrote on Sun, 23 June 2013 05:52

Interesting,

 

I consistently get results showing that strjoin() is about twice as

fast

as the concatenate.  This is on a 3GHz Pentium running Linux 2.6.34.

But as you mentioned, it's miniscule compared to the time it takes to

execute the script.

 

Hi Chuck

My previously mentioned times were from a P500 running Win2000. I use that

PC for writing the ULP as it shows up bad coding choices. If it's fast on

that machine then its acceptable.

I timed it again using a Win7 32bit Dual Core mailto:T6670@2.20GHz

On the first pass the strJoin was 82% of the Concatenation time

Subsequent passes averaged 65% of the Concatenation time and quite

consistant.

So although strJoin took longer on the first pass it was always faster than

concatenation.

Warren

--

Web access to CadSoft support forums at www.eaglecentral.ca.  Where the CadSoft EAGLE community meets.

Chuck Huber wrote on Sun, 23 June 2013 05:52

Interesting,

 

I consistently get results showing that strjoin() is about twice as

fast

as the concatenate.  This is on a 3GHz Pentium running Linux 2.6.34.

But as you mentioned, it's miniscule compared to the time it takes to

execute the script.

 

Hi Chuck

My previously mentioned times were from a P500 running Win2000. I use that

PC for writing the ULP as it shows up bad coding choices. If it's fast on

that machine then its acceptable.

I timed it again using a Win7 32bit Dual Core mailto:T6670@2.20GHz

On the first pass the strJoin was 82% of the Concatenation time

Subsequent passes averaged 65% of the Concatenation time and quite

consistant.

So although strJoin took longer on the first pass it was always faster than

concatenation.

Warren

--

Web access to CadSoft support forums at www.eaglecentral.ca.  Where the CadSoft EAGLE community meets.

Il 24/06/2013 02:30, Warren Brayshaw ha scritto:

I timed it again using a Win7 32bit Dual Core mailto:T6670@2.20GHz

 

On the first pass the strJoin was 82% of the Concatenation time

 

Subsequent passes averaged 65% of the Concatenation time and quite

consistant.

 

So although strJoin took longer on the first pass it was always faster than

concatenation.

Hi Warren,

First, thanks for writing the test ULP to time the 2 solutions!

I can confirm your results, the first run gives 91% and second 73% speed

between strjoin vs concatenation. With 100.000 loops I get times on the

order of 60 to 70ms, running on Core2-Duo 3.16GHz Win-7/64bit.

Now, I suggest to broaden the test cases a little bit, here's why:

Often there is a need to build more complex commands, using sprintf.

On some system ULPs I can see code like this:

=============

string Cmd, h;

sprintf(h, "EDIT %s.PaC;\nGRID %s;\n", s[0], Unit);

Cmd += h;

=============

In this kind of cases sprintf() formatting becomes beneficial, else you

have to write something like following (hardly readable):

=============

Cmd = "EDIT "s[0]".PaC;\nGRID "Unit+";\n";

=============

If I modify your test ULP inside the loops like this:

=============

for(i=1; i<100000; i++){

     string s;

     sprintf(s, " %s", words[i & 3]);

     runtext += s;

}

for(i=1; i<100000; i++){

     sprintf(a[i], " %s", words[i & 3]);

}

=============

I get some AMAZING speed differences.

In this test code strjoin consumes 250ms and concatenation 20940ms. The

strjoin vs concatenation ratio becomes 1,2%, in favor of strjoin!!!

Am I doing or measuring something totally wrong?

Kim

KimP wrote:

Il 24/06/2013 02:30, Warren Brayshaw ha scritto:

I timed it again using a Win7 32bit Dual Core mailto:T6670@2.20GHz

 

On the first pass the strJoin was 82% of the Concatenation time

 

Subsequent passes averaged 65% of the Concatenation time and quite

consistant.

 

So although strJoin took longer on the first pass it was always

faster than concatenation.

 

Hi Warren,

 

First, thanks for writing the test ULP to time the 2 solutions!

I can confirm your results, the first run gives 91% and second 73%

speed between strjoin vs concatenation. With 100.000 loops I get

times on the order of 60 to 70ms, running on Core2-Duo 3.16GHz

Win-7/64bit.

 

Now, I suggest to broaden the test cases a little bit, here's why:

Often there is a need to build more complex commands, using sprintf.

On some system ULPs I can see code like this:

=============

string Cmd, h;

sprintf(h, "EDIT %s.PaC;\nGRID %s;\n", s[0], Unit);

Cmd += h;

=============

 

In this kind of cases sprintf() formatting becomes beneficial, else

you have to write something like following (hardly readable):

=============

Cmd = "EDIT "s[0]".PaC;\nGRID "Unit+";\n";

=============

 

If I modify your test ULP inside the loops like this:

=============

for(i=1; i<100000; i++){

string s;

sprintf(s, " %s", words[i & 3]);

runtext += s;

}

 

for(i=1; i<100000; i++){

sprintf(a[i], " %s", words[i & 3]);

}

=============

I get some AMAZING speed differences.

In this test code strjoin consumes 250ms and concatenation 20940ms.

The strjoin vs concatenation ratio becomes 1,2%, in favor of

strjoin!!!

Am I doing or measuring something totally wrong?

 

Kim

That's very interesting.

It took me a while to figure out where the huge time difference was coming

from.

The answer is the difference in time it takes Eagle to fetch a string

compared to an array element.

string a[] = {"test"};

string b = "test";

runtext += b;

runtext += a[0];

These run at very different speeds.

--

Viewed / responded via the newsgroup at

news.cadsoft.de

The answer is the difference in time it takes Eagle to fetch a string

compared to an array element.

 

string a[] = {"test"};

string b = "test";

 

runtext += b;

runtext += a[0];

 

These run at very different speeds.

Good finding(s), never thought there would be such a huge difference

between string handling (variable vs array). Although, in real-world

ULPs I don't think these details make such a big difference, they might

reveal some areas for improvement.

In coding ULPs I believe more performance is gained by good and simple

code+data structures. I have not run into situations where ULPs run for

endless seconds/minutes, but can imagine some possible cases (example

netlist/footprint conversions) where the ULP execution time builds up

and becomes critical.

Kim

On 06/25/2013 05:13 AM, KimP wrote:

Good finding(s), never thought there would be such a huge difference

between string handling (variable vs array). Although, in real-world

ULPs I don't think these details make such a big difference, they

might reveal some areas for improvement.

Jorge,  Would it be possible to ask one of the developers to provide

some enlightenment on how strings and string arrays are handled?

Thanks,

    - Chuck

On 6/25/2013 8:56 AM, Chuck Huber wrote:

On 06/25/2013 05:13 AM, KimP wrote:

 

Good finding(s), never thought there would be such a huge difference

between string handling (variable vs array). Although, in real-world

ULPs I don't think these details make such a big difference, they

might reveal some areas for improvement.

 

Jorge,  Would it be possible to ask one of the developers to provide

some enlightenment on how strings and string arrays are handled?

 

Thanks,

     - Chuck

 

Hi Chuck,

I'll see what I can do. I've tried the ULP on my linux box and the

strjoin operation was always faster than the concatenation. However both

were at least an order of magnitude slower than writing the script file.

I'm sure they'll be very interested in this experiment you guys have

carried out.

Let me know if there's anything else I can do for you.

Best Regards,

Jorge Garcia

Hi,

On 06/25/13 20:04, Jorge Garcia wrote:

On 6/25/2013 8:56 AM, Chuck Huber wrote:

On 06/25/2013 05:13 AM, KimP wrote:

 

Good finding(s), never thought there would be such a huge difference

between string handling (variable vs array). Although, in real-world

ULPs I don't think these details make such a big difference, they

might reveal some areas for improvement.

 

Jorge,  Would it be possible to ask one of the developers to provide

some enlightenment on how strings and string arrays are handled?

 

Thanks,

     - Chuck

 

Hi Chuck,

 

I'll see what I can do. I've tried the ULP on my linux box and the

strjoin operation was always faster than the concatenation. However both

were at least an order of magnitude slower than writing the script file.

 

I'm sure they'll be very interested in this experiment you guys have

carried out.

 

Let me know if there's anything else I can do for you.

 

Best Regards,

Jorge Garcia

if you have a loop that does a concatenation in each step like

"

string Result;

string Lines[]; // say 100000 lines filled with some strings

for (int i=0; i<100000; ++i)

   Result += Lines[i];

"

then in each step a reallocation occurs for Result which takes time.

In that case strjoin is much better:

"

Result = strjoin(Lines,'');

"

strjoin does the whole thing in one step, only one reallocation and copy.

Hope this makes it clearer.

Regards,

Walter Spermann

--

-

Walter Spermann

Softwareentwicklung

CadSoft Computer GmbH

Pleidolfweg 15

84568 Pleiskirchen

Tel.: 08635/6989-10

www.cadsoft.de

-

Registergericht: Amtsgericht Traunstein HRB 5573

Geschäftsführer: Thomas Liratsch

-

Walter Spermann wrote on Thu, 27 June 2013 05:40

Hi,

 

]

if you have a loop that does a concatenation in each step like

"

string Result;

string Lines[]; // say 100000 lines filled with some strings

for (int i=0; i<100000; ++i)

   Result += Lines[i];

"

then in each step a reallocation occurs for Result which takes time.

In that case strjoin is much better:

"

Result = strjoin(Lines,'');

"

strjoin does the whole thing in one step, only one reallocation and

copy.

Hope this makes it clearer.

Thanks for the information Walter.

To all.

I have had a further play. I wondered how the final string size affected

performance. I found once the size exceeded 132KB the times increased

dramatically. The graph headed for the heavens.

That size string would not be common so is probably not relevant

sprintf() has a far more dramatic effect. The attached ULP illustrates

this.

So, in some ULPs it would be worth using sprintf() sparingly.

I noticed when running the ULP that the reported times varied quite a bit

with either of the top two sprintf techniques  being the faster. The string

size remained the same so I can only imagine there is contention for

resources when this is observed. 

Regards

Warren

--

Web access to CadSoft support forums at www.eaglecentral.ca.  Where the CadSoft EAGLE community meets.