Shay Ginsbourg | 12 Jun 2012 10:56
Picon

TCP Sampler Read Exception

Hi all,

Can anyone suggest what does the following error mean ?

After each TCP sampler, we receive "Response code: 500" and

"Response message: org.apache.jmeter.protocol.tcp.sampler.ReadException:"

Any help would be greatly appreciated.

thanks,
Shay
sebb | 12 Jun 2012 11:02
Picon

Re: TCP Sampler Read Exception

On 12 June 2012 09:56, Shay Ginsbourg <sginsbourg <at> gmail.com> wrote:
> Hi all,
>
> Can anyone suggest what does the following error mean ?
>
> After each TCP sampler, we receive "Response code: 500" and
>
> "Response message: org.apache.jmeter.protocol.tcp.sampler.ReadException:"
>
> Any help would be greatly appreciated.

Most likely the read terminated unexpectedly:

http://jmeter.apache.org/api/org/apache/jmeter/protocol/tcp/sampler/TCPClient.html#read%28java.io.InputStream%29

> thanks,
> Shay
Kirk Pepperdine | 12 Jun 2012 17:31
Picon

JMeter threading model

Hi,

I guess this is mostly directed towards Sebb.. I spent a bit looking at JMeter performance today. What I
found was that the thread group starts all of it's threads at the beginning and then uses them over time. Is
there a way to easily change this so that the thread group only creates threads on demand?

Regards,
Kirk
Shmuel Krakower | 12 Jun 2012 18:28
Picon

Re: JMeter threading model

Hi,
I was thinking to suggest working with JP <at> GC Ultimate thread group, but it
behaves the same (pre-creating the threads).
Kirk, why does it matter to you?

Regards,
Shmuel.

On Tue, Jun 12, 2012 at 6:31 PM, Kirk Pepperdine
<kirk.pepperdine <at> gmail.com>wrote:

> Hi,
>
> I guess this is mostly directed towards Sebb.. I spent a bit looking at
> JMeter performance today. What I found was that the thread group starts all
> of it's threads at the beginning and then uses them over time. Is there a
> way to easily change this so that the thread group only creates threads on
> demand?
>
> Regards,
> Kirk
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe <at> jmeter.apache.org
> For additional commands, e-mail: user-help <at> jmeter.apache.org
>
>
Philippe Mouawad | 12 Jun 2012 19:46
Picon

Re: JMeter threading model

Hello M. Pepperdine,
There is no way today to do that.
There is an enhancement request that has not been implemented yet although
started some time ago:

   - https://issues.apache.org/bugzilla/show_bug.cgi?id=47886

JMeter core is build around the paradigm:

   - 1 user == 1 Thread
   - And lot of its internal behaviour makes this supposition

This could be changed in 2 ways:

   - 1) use for example a ThreadPoolExecutor without limiting its growth to
   avoid thread starvation during a Load Test but to create as little threads
   as possible
   - 2) Use a completely different approach based on something like netty
   but this one would be a revolution and a too big impact in my opinion

But even with 1) we have to handle the ThreadLocal objects currently used
so this need through investigation.

Regards
Philippe M.
http://www.ubik-ingenierie.com

On Tue, Jun 12, 2012 at 6:28 PM, Shmuel Krakower <shmulikk <at> gmail.com> wrote:

> Hi,
(Continue reading)

Kirk Pepperdine | 12 Jun 2012 23:06
Picon

Re: JMeter threading model

Hi,

I figured thread pooling would be revolutionary so I wasn't suggesting that. I would be very useful just
delay the creation of a thread until it was asked for.

Regards,
Kirk

On 2012-06-12, at 8:46 PM, Philippe Mouawad wrote:

> Hello M. Pepperdine,
> There is no way today to do that.
> There is an enhancement request that has not been implemented yet although
> started some time ago:
> 
>   - https://issues.apache.org/bugzilla/show_bug.cgi?id=47886
> 
> JMeter core is build around the paradigm:
> 
>   - 1 user == 1 Thread
>   - And lot of its internal behaviour makes this supposition
> 
> This could be changed in 2 ways:
> 
>   - 1) use for example a ThreadPoolExecutor without limiting its growth to
>   avoid thread starvation during a Load Test but to create as little threads
>   as possible
>   - 2) Use a completely different approach based on something like netty
>   but this one would be a revolution and a too big impact in my opinion
> 
(Continue reading)

sebb | 12 Jun 2012 23:54
Picon

Re: JMeter threading model

On 12 June 2012 22:06, Kirk Pepperdine <kirk.pepperdine <at> gmail.com> wrote:
> Hi,
>
> I figured thread pooling would be revolutionary so I wasn't suggesting that. I would be very useful just
delay the creation of a thread until it was asked for.

Not sure I understand how it would help to delay the thread creation,
except perhaps for the case where the first threads have finished
processing by the time the last threads start running samples.

> Regards,
> Kirk
>
> On 2012-06-12, at 8:46 PM, Philippe Mouawad wrote:
>
>> Hello M. Pepperdine,
>> There is no way today to do that.
>> There is an enhancement request that has not been implemented yet although
>> started some time ago:
>>
>>   - https://issues.apache.org/bugzilla/show_bug.cgi?id=47886
>>
>> JMeter core is build around the paradigm:
>>
>>   - 1 user == 1 Thread
>>   - And lot of its internal behaviour makes this supposition
>>
>> This could be changed in 2 ways:
>>
>>   - 1) use for example a ThreadPoolExecutor without limiting its growth to
(Continue reading)

Kirk Pepperdine | 12 Jun 2012 23:57
Picon

Re: JMeter threading model


On 2012-06-13, at 12:54 AM, sebb wrote:

> On 12 June 2012 22:06, Kirk Pepperdine <kirk.pepperdine <at> gmail.com> wrote:
>> Hi,
>> 
>> I figured thread pooling would be revolutionary so I wasn't suggesting that. I would be very useful just
delay the creation of a thread until it was asked for.
> 
> Not sure I understand how it would help to delay the thread creation,
> except perhaps for the case where the first threads have finished
> processing by the time the last threads start running samples.

Bingo!!! ;-)

Kirk
sebb | 13 Jun 2012 00:02
Picon

Re: JMeter threading model

On 12 June 2012 22:57, Kirk Pepperdine <kirk.pepperdine <at> gmail.com> wrote:
>
> On 2012-06-13, at 12:54 AM, sebb wrote:
>
>> On 12 June 2012 22:06, Kirk Pepperdine <kirk.pepperdine <at> gmail.com> wrote:
>>> Hi,
>>>
>>> I figured thread pooling would be revolutionary so I wasn't suggesting that. I would be very useful just
delay the creation of a thread until it was asked for.
>>
>> Not sure I understand how it would help to delay the thread creation,
>> except perhaps for the case where the first threads have finished
>> processing by the time the last threads start running samples.
>
> Bingo!!! ;-)

So what percentage of use cases need to follow this model?

Most of the JMeter testing I have done was long running tests where
all threads were active for most of the run.

> Kirk
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe <at> jmeter.apache.org
> For additional commands, e-mail: user-help <at> jmeter.apache.org
>
Robin D. Wilson | 13 Jun 2012 00:42
Picon

RE: JMeter threading model

Currently, JMeter will run 100 threads, for 100 iterations each in order to reach 10,000 total passes
through my test case. If
thread 1 completes its 100 iterations before any of the rest of the threads, it terminates and then I only
have 99 threads running
until the 10,000 iterations complete. My test will show a performance change at the end - as the number of
total simultaneous
threads begins to trail off, the performance will appear to improve. For example, I will show the sampler as
having completed 9200
requests when the first thread finishes its 100 iterations. This means for the remaining 800 requests, I
only have a maximum 99
threads operating. Then at 9300 samples, 19 more threads might have died off - so I only have 80 threads max
operating for the
remaining 700 requests. As you can imagine, this means that by the end of the test - the performance numbers
will start to
progressively improve (since fewer threads means less workload on the process being measured, and
therefore faster response times).

It would be really nice if JMeter just kept a pool of 100 threads operating on requests for the duration of the
10,000 iterations,
so that threads would only die off during the final iteration, leaving the server at more-or-less peak load
throughout the test.

From a code standpoint, this doesn't seem like it would be too hard to setup - just identify how many total
iterations need to be
run through the thread group, startup the total number of threads you need, and let each thread keep going
until all the iterations
have been started. (Of course, I say that knowing that I'm just a 'manager' type, and won't be coding it myself...)

--
Robin D. Wilson
(Continue reading)

Deepak Shetty | 13 Jun 2012 00:47
Picon

Re: JMeter threading model

>My test will show a performance change at the end - as the number of total
simultaneous threads begins to trail off, the performance will appear to
improve.
If so , your test needs more than 1 machine to run and the threads on
demand wont really help your use case.

On Tue, Jun 12, 2012 at 3:42 PM, Robin D. Wilson <rwilson2 <at> gmail.com> wrote:

> Currently, JMeter will run 100 threads, for 100 iterations each in order
> to reach 10,000 total passes through my test case. If
> thread 1 completes its 100 iterations before any of the rest of the
> threads, it terminates and then I only have 99 threads running
> until the 10,000 iterations complete. My test will show a performance
> change at the end - as the number of total simultaneous
> threads begins to trail off, the performance will appear to improve. For
> example, I will show the sampler as having completed 9200
> requests when the first thread finishes its 100 iterations. This means for
> the remaining 800 requests, I only have a maximum 99
> threads operating. Then at 9300 samples, 19 more threads might have died
> off - so I only have 80 threads max operating for the
> remaining 700 requests. As you can imagine, this means that by the end of
> the test - the performance numbers will start to
> progressively improve (since fewer threads means less workload on the
> process being measured, and therefore faster response times).
>
> It would be really nice if JMeter just kept a pool of 100 threads
> operating on requests for the duration of the 10,000 iterations,
> so that threads would only die off during the final iteration, leaving the
> server at more-or-less peak load throughout the test.
>
(Continue reading)

Robin D. Wilson | 13 Jun 2012 01:01
Picon

RE: JMeter threading model

I'm not sure I follow... 

My JMeter test runs on a single Win7 box. My test environment consists of a web server, multiple Java App
servers and a database
server. The performance appears to improve because I'm hitting the test environment with fewer and fewer
threads as the test begins
to wrap up. I'm just saying that it would be nice if I could delay that drop-off until I get to the last 100
samples, rather than
several hundred samples before the test ends. It probably doesn't make a big difference to me - since I'm
really just comparing the
results of the last time I ran the test against the results of this time. But it would make my test a better
measure of operating
characteristics of my system 'under load' to have it consistently maintain the full number of requesting
threads throughout the
test.

--
Robin D. Wilson
Sr. Director of Web Development
KingsIsle Entertainment, Inc.
VOICE: 512-777-1861
www.KingsIsle.com

-----Original Message-----
From: Deepak Shetty [mailto:shettyd <at> gmail.com] 
Sent: Tuesday, June 12, 2012 5:47 PM
To: JMeter Users List
Subject: Re: JMeter threading model

>My test will show a performance change at the end - as the number of total
(Continue reading)

Deepak Shetty | 13 Jun 2012 01:10
Picon

Re: JMeter threading model

sorry i think i misread your email.

>But it would make my test a better measure of operating
>characteristics of my system 'under load' to have it consistently maintain
the full number of requesting threads throughout the
>test.
Cant you already do this? fix the duration of the test instead of the
number of times you want to run .
still seems unrelated to kirk's request :)

On Tue, Jun 12, 2012 at 4:01 PM, Robin D. Wilson <rwilson2 <at> gmail.com> wrote:

> I'm not sure I follow...
>
> My JMeter test runs on a single Win7 box. My test environment consists of
> a web server, multiple Java App servers and a database
> server. The performance appears to improve because I'm hitting the test
> environment with fewer and fewer threads as the test begins
> to wrap up. I'm just saying that it would be nice if I could delay that
> drop-off until I get to the last 100 samples, rather than
> several hundred samples before the test ends. It probably doesn't make a
> big difference to me - since I'm really just comparing the
> results of the last time I ran the test against the results of this time.
> But it would make my test a better measure of operating
> characteristics of my system 'under load' to have it consistently maintain
> the full number of requesting threads throughout the
> test.
>
> --
> Robin D. Wilson
(Continue reading)

sebb | 13 Jun 2012 01:06
Picon

Re: JMeter threading model

On 12 June 2012 23:42, Robin D. Wilson <rwilson2 <at> gmail.com> wrote:
> Currently, JMeter will run 100 threads, for 100 iterations each in order to reach 10,000 total passes
through my test case. If
> thread 1 completes its 100 iterations before any of the rest of the threads, it terminates and then I only
have 99 threads running
> until the 10,000 iterations complete. My test will show a performance change at the end - as the number of
total simultaneous
> threads begins to trail off, the performance will appear to improve. For example, I will show the sampler
as having completed 9200
> requests when the first thread finishes its 100 iterations. This means for the remaining 800 requests, I
only have a maximum 99
> threads operating. Then at 9300 samples, 19 more threads might have died off - so I only have 80 threads max
operating for the
> remaining 700 requests. As you can imagine, this means that by the end of the test - the performance numbers
will start to
> progressively improve (since fewer threads means less workload on the process being measured, and
therefore faster response times).
>
> It would be really nice if JMeter just kept a pool of 100 threads operating on requests for the duration of
the 10,000 iterations,
> so that threads would only die off during the final iteration, leaving the server at more-or-less peak
load throughout the test.

JMeter can do this already.
Rather than specify a fixed number of loops, use a CSV Config file
with 10,000 entries, and set it to stop the thread (not the test, as
that would not let some threads complete) when EOF is reached.

For tests where a CSV data file is not appropriate or convenient, it
might make sense to implement a feature which counts total loops and
(Continue reading)

Robin D. Wilson | 13 Jun 2012 01:39
Picon

RE: JMeter threading model

sebb wrote:
>For tests where a CSV data file is not appropriate or convenient, it
>might make sense to implement a feature which counts total loops and
>causes the test to stop when a pre-specified limit is reached. Perhaps
>at thread group level, which already has a duration limit. Or a test
>element of some kind.

At the thread group level is exactly what I'm suggesting...

In the meantime, I will setup a csv file with 0-9999 in it (one number per row), and see if I can get that to work
like you suggest.
Seems a little clunky, but if it works - no issue.

--
Robin D. Wilson
Sr. Director of Web Development
KingsIsle Entertainment, Inc.
VOICE: 512-777-1861
www.KingsIsle.com

-----Original Message-----
From: sebb [mailto:sebbaz <at> gmail.com] 
Sent: Tuesday, June 12, 2012 6:07 PM
To: JMeter Users List
Subject: Re: JMeter threading model

On 12 June 2012 23:42, Robin D. Wilson <rwilson2 <at> gmail.com> wrote:
> Currently, JMeter will run 100 threads, for 100 iterations each in order to reach 10,000 total passes
through my test case. If
> thread 1 completes its 100 iterations before any of the rest of the threads, it terminates and then I only
(Continue reading)

sebb | 13 Jun 2012 01:55
Picon

Re: JMeter threading model

On 13 June 2012 00:39, Robin D. Wilson <rwilson2 <at> gmail.com> wrote:
> sebb wrote:
>>For tests where a CSV data file is not appropriate or convenient, it
>>might make sense to implement a feature which counts total loops and
>>causes the test to stop when a pre-specified limit is reached. Perhaps
>>at thread group level, which already has a duration limit. Or a test
>>element of some kind.
>
> At the thread group level is exactly what I'm suggesting...
>
> In the meantime, I will setup a csv file with 0-9999 in it (one number per row), and see if I can get that to work
like you suggest.

I would start with a shorter file and simple test plan ...

> Seems a little clunky, but if it works - no issue.

Just realised, another possibility is:

Thread Group
+ If Controller, condition: ${__counter(FALSE)} > 10000
+ + Test Action/Stop Thread.
+ rest of thread samples

As the Perl mantra goes: TMTOWTDI

> --
> Robin D. Wilson
> Sr. Director of Web Development
> KingsIsle Entertainment, Inc.
(Continue reading)

Anthony Johnson | 13 Jun 2012 02:14
Picon

Re: JMeter threading model

On Tue, Jun 12, 2012 at 7:06 PM, sebb <sebbaz <at> gmail.com> wrote:
>
> On 12 June 2012 23:42, Robin D. Wilson <rwilson2 <at> gmail.com> wrote:
> > Currently, JMeter will run 100 threads, for 100 iterations each in order
> > to reach 10,000 total passes through my test case. If
> > thread 1 completes its 100 iterations before any of the rest of the
> > threads, it terminates and then I only have 99 threads running
> > until the 10,000 iterations complete. My test will show a performance
> > change at the end - as the number of total simultaneous
> > threads begins to trail off, the performance will appear to improve. For
> > example, I will show the sampler as having completed 9200
> > requests when the first thread finishes its 100 iterations. This means
> > for the remaining 800 requests, I only have a maximum 99
> > threads operating. Then at 9300 samples, 19 more threads might have died
> > off - so I only have 80 threads max operating for the
> > remaining 700 requests. As you can imagine, this means that by the end
> > of the test - the performance numbers will start to
> > progressively improve (since fewer threads means less workload on the
> > process being measured, and therefore faster response times).
> >
> > It would be really nice if JMeter just kept a pool of 100 threads
> > operating on requests for the duration of the 10,000 iterations,
> > so that threads would only die off during the final iteration, leaving
> > the server at more-or-less peak load throughout the test.
>
> JMeter can do this already.
> Rather than specify a fixed number of loops, use a CSV Config file
> with 10,000 entries, and set it to stop the thread (not the test, as
> that would not let some threads complete) when EOF is reached.
>
(Continue reading)

sebb | 13 Jun 2012 02:24
Picon

Re: JMeter threading model

On 13 June 2012 01:14, Anthony Johnson <ansoni <at> gmail.com> wrote:
> On Tue, Jun 12, 2012 at 7:06 PM, sebb <sebbaz <at> gmail.com> wrote:
>>
>> On 12 June 2012 23:42, Robin D. Wilson <rwilson2 <at> gmail.com> wrote:
>> > Currently, JMeter will run 100 threads, for 100 iterations each in order
>> > to reach 10,000 total passes through my test case. If
>> > thread 1 completes its 100 iterations before any of the rest of the
>> > threads, it terminates and then I only have 99 threads running
>> > until the 10,000 iterations complete. My test will show a performance
>> > change at the end - as the number of total simultaneous
>> > threads begins to trail off, the performance will appear to improve. For
>> > example, I will show the sampler as having completed 9200
>> > requests when the first thread finishes its 100 iterations. This means
>> > for the remaining 800 requests, I only have a maximum 99
>> > threads operating. Then at 9300 samples, 19 more threads might have died
>> > off - so I only have 80 threads max operating for the
>> > remaining 700 requests. As you can imagine, this means that by the end
>> > of the test - the performance numbers will start to
>> > progressively improve (since fewer threads means less workload on the
>> > process being measured, and therefore faster response times).
>> >
>> > It would be really nice if JMeter just kept a pool of 100 threads
>> > operating on requests for the duration of the 10,000 iterations,
>> > so that threads would only die off during the final iteration, leaving
>> > the server at more-or-less peak load throughout the test.
>>
>> JMeter can do this already.
>> Rather than specify a fixed number of loops, use a CSV Config file
>> with 10,000 entries, and set it to stop the thread (not the test, as
>> that would not let some threads complete) when EOF is reached.
(Continue reading)

Kirk Pepperdine | 13 Jun 2012 07:05
Picon

Re: JMeter threading model

Hi Sebb,

We've had this conversation before and I did some preliminary work to setup a different type of thread group
but the couplings between the existing thread group and the model meant that an extensive refactoring
would be involved. Since that involves a *lot* more than just a simple plugin...

So, the current implementation supports a closed system model meaning, rate of entry into the system
equals rate of exit from the system. This is exactly what you want if you're load testing a call centre where
the number of servers (operators) is fixed and gate entry into the system. However, I'm often simulating
open systems which means I do not want rate of entry into the system to be controlled by the performance of
the system (rate of exit). More over, those that attend my performance tuning seminars come to understand
why this is an important aspect of getting your test environment right and test harness correctly setup as
it can adversely affect the quality of your test which can and often does, change the results of the test.

As an example, today I will show a group how to tune an application by a partner company. That application has
a number of "performance problems" backed into it. If I use the traditional means of using JMeter I will
find a different set of performance issues than if I load with a pattern that is similar that found in
production. In other words, with this particular application, JMeter exposes "problems" that are
artifacts of how it wants to inject load on a system. I can fix all of these problems and eventually I'll get
to a point where I'll fix everything that needs to be fixed. That said, if I can coerce JMeter to load as an
open system I'll get to the problems without having to fix the artifacts (the things that really don't need
fixing). To coerce JMeter into being an open system requires one to use a large number of very short lived
threads. So I may only have 400-500 active threads at any point in time but in order to achieve that load over
a 1 or two hour test I may have to specify 10s of thousands of threads. Since all of the threads are created up
front, this simply doesn't work.

You might ask why not just specify 400-500 threads and loop over them? in theory you'd think it would work but
as you tune the system and the performance characteristics change. Going back to the baked application,
before I start tuning, the active user count is several thousand. In other words, the tuned system is
better able to clear users out and that changes the performance profile in a way that hard to emulate with
(Continue reading)

sebb | 13 Jun 2012 09:58
Picon

Re: JMeter threading model

On 13 June 2012 06:05, Kirk Pepperdine <kirk.pepperdine <at> gmail.com> wrote:
> Hi Sebb,
>
> We've had this conversation before and I did some preliminary work to setup a different type of thread
group but the couplings between the existing thread group and the model meant that an extensive
refactoring would be involved. Since that involves a *lot* more than just a simple plugin...
>
> So, the current implementation supports a closed system model meaning, rate of entry into the system
equals rate of exit from the system. This is exactly what you want if you're load testing a call centre where
the number of servers (operators) is fixed and gate entry into the system. However, I'm often simulating
open systems which means I do not want rate of entry into the system to be controlled by the performance of
the system (rate of exit). More over, those that attend my performance tuning seminars come to understand
why this is an important aspect of getting your test environment right and test harness correctly setup as
it can adversely affect the quality of your test which can and often does, change the results of the test.
>
> As an example, today I will show a group how to tune an application by a partner company. That application
has a number of "performance problems" backed into it. If I use the traditional means of using JMeter I will
find a different set of performance issues than if I load with a pattern that is similar that found in
production. In other words, with this particular application, JMeter exposes "problems" that are
artifacts of how it wants to inject load on a system. I can fix all of these problems and eventually I'll get
to a point where I'll fix everything that needs to be fixed. That said, if I can coerce JMeter to load as an
open system I'll get to the problems without having to fix the artifacts (the things that really don't need
fixing). To coerce JMeter into being an open system requires one to use a large number of very short lived
threads. So I may only have 400-500 active threads at any point in time but in order to achieve that load over
a 1 or two hour test I may have to specify 10s of thousands of threads. Since all of the threads are created up
front, this simply doesn't work.
>
> You might ask why not just specify 400-500 threads and loop over them? in theory you'd think it would work
but as you tune the system and the performance characteristics change. Going back to the baked
application, before I start tuning, the active user count is several thousand. In other words, the tuned
(Continue reading)

Flavio Cysne | 13 Jun 2012 13:56
Picon

Re: JMeter threading model

Hi Kirk,

    just a thought on this case, and if you start as many JMeter instances
(using servers or not) as needed, once at a time, to increase the load,
let's say, pseudo-dynamically?

    I think this way you could, in some way, achieve the desired load. The
"hard work" will be on merging jtl files in only one big jtl file to
generate reports.

    I have to agree with you that increasing/decreasing dynamically the
load is better than starting many tests to pin-point a spot on the graph.

Hope it helps you.
Flávio Cysne

2012/6/13 Kirk Pepperdine <kirk.pepperdine <at> gmail.com>

> Hi Sebb,
>
> We've had this conversation before and I did some preliminary work to
> setup a different type of thread group but the couplings between the
> existing thread group and the model meant that an extensive refactoring
> would be involved. Since that involves a *lot* more than just a simple
> plugin...
>
> So, the current implementation supports a closed system model meaning,
> rate of entry into the system equals rate of exit from the system. This is
> exactly what you want if you're load testing a call centre where the number
> of servers (operators) is fixed and gate entry into the system. However,
(Continue reading)

Philippe Mouawad | 14 Jun 2012 22:30
Picon

Re: JMeter threading model

Hi M. Pepperdine,

On Wed, Jun 13, 2012 at 7:05 AM, Kirk Pepperdine
<kirk.pepperdine <at> gmail.com>wrote:

> Hi Sebb,
>
> We've had this conversation before and I did some preliminary work to
> setup a different type of thread group but the couplings between the
> existing thread group and the model meant that an extensive refactoring
> would be involved. Since that involves a *lot* more than just a simple
> plugin...
>
> So, the current implementation supports a closed system model meaning,
> rate of entry into the system equals rate of exit from the system.This is
> exactly what you want if you're load testing a call centre where the number
> of servers (operators) is fixed and gate entry into the system. However,
> I'm often simulating open systems which means I do not want rate of entry
> into the system to be controlled by the performance of the system (rate of
> exit).

What makes you think JMeter does this ?

> More over, those that attend my performance tuning seminars come to
> understand why this is an important aspect of getting your test environment
> right and test harness correctly setup as it can adversely affect the
> quality of your test which can and often does, change the results of the
> test.
>

(Continue reading)

Philippe Mouawad | 14 Jun 2012 23:13
Picon

Re: JMeter threading model

Bugzilla created:

   - https://issues.apache.org/bugzilla/show_bug.cgi?id=53418

Regards

Philippe M.

On Thu, Jun 14, 2012 at 10:30 PM, Philippe Mouawad <
philippe.mouawad <at> gmail.com> wrote:

> Hi M. Pepperdine,
>
> On Wed, Jun 13, 2012 at 7:05 AM, Kirk Pepperdine <
> kirk.pepperdine <at> gmail.com> wrote:
>
>> Hi Sebb,
>>
>> We've had this conversation before and I did some preliminary work to
>> setup a different type of thread group but the couplings between the
>> existing thread group and the model meant that an extensive refactoring
>> would be involved. Since that involves a *lot* more than just a simple
>> plugin...
>>
>> So, the current implementation supports a closed system model meaning,
>> rate of entry into the system equals rate of exit from the system.This is
>> exactly what you want if you're load testing a call centre where the number
>> of servers (operators) is fixed and gate entry into the system. However,
>> I'm often simulating open systems which means I do not want rate of entry
>> into the system to be controlled by the performance of the system (rate of
(Continue reading)

Kirk Pepperdine | 15 Jun 2012 07:24
Picon

Re: JMeter threading model

Hi,

I'm very happy to add details if need be.

Regards,
Kirk

On 2012-06-15, at 12:13 AM, Philippe Mouawad wrote:

> Bugzilla created:
> 
>   - https://issues.apache.org/bugzilla/show_bug.cgi?id=53418
> 
> 
> Regards
> 
> Philippe M.
> 
> On Thu, Jun 14, 2012 at 10:30 PM, Philippe Mouawad <
> philippe.mouawad <at> gmail.com> wrote:
> 
>> Hi M. Pepperdine,
>> 
>> On Wed, Jun 13, 2012 at 7:05 AM, Kirk Pepperdine <
>> kirk.pepperdine <at> gmail.com> wrote:
>> 
>>> Hi Sebb,
>>> 
>>> We've had this conversation before and I did some preliminary work to
>>> setup a different type of thread group but the couplings between the
(Continue reading)

D G | 15 Jun 2012 10:06
Picon
Gravatar

Re: JMeter threading model

Details are very welcome as I'm still not sure whether I fully
understand the problem :)

Thread pooling does sound nice and it might enable a feature I'd like:

On-demand creation of threads. Say you're looping 10 threads, system
is under no load.
Currently: if you want 10 more users you need to stop the test, add 10
users and start it again.

It would be nice if it where possible to simply say '+10', followed by
some monitoring and then another '+10' (30 threads running total) etc.
(10 is an example number, +/- n users would be the goal)
Mostly for the 'getting a feel for the system' part of the test, the
initial setup of it all.

Regards,
Daniel

On Fri, Jun 15, 2012 at 7:24 AM, Kirk Pepperdine
<kirk.pepperdine <at> gmail.com> wrote:
> Hi,
>
> I'm very happy to add details if need be.
>
> Regards,
> Kirk
>
> On 2012-06-15, at 12:13 AM, Philippe Mouawad wrote:
>
(Continue reading)

Flavio Cysne | 15 Jun 2012 20:18
Picon

Re: JMeter threading model

I have tested something in JMeter that could achieve a dynamic threading
model and there was no need of new implementations.

Basically, I have this structure:

1. Test Plan: { variables : [ {name: "maxThreads", value: 5}, {name:
"currentThreads", value: 1} ] }
2. | - Thread Group: { threads: "${maxThreads}", rampup: 0, infinite: true }
3.     | - If Controller: { condition: "(${__threadNum()} ===
${maxThreads})" }
4.         | - Sampler /* I've used HTTP Request, but anyone would be used
*/
5.             | - RegEx Extractor: {refName: "threads", expression:
"(.*)", model: "$1$", matchNo: "1", defaultValue: "0"}
6.             | - BeanShell Post-Processor: {language: "javascript",
script: "var threads = vars.get(\"threads\");\nprops.put(\"currentThreads
\",threads);"}
7.     | - If Controller: { condition: "(${__threadNum()} < ${maxThreads})"
}
8.         | - If Controller: { condition: "(${__threadNum()} <= ${__P(
currentThreads,0)})" }
9.             | - All other Samplers go here

The idea is starting the maximum number of desired threads at the beginning
but only use what you want.
The first "If Controller" is executed and update a property named "threads"
with the value retrieved from a response (HTTP, SOAP, file, whatever JMeter
can access)
When the first loop iteration is done line 9 will be skipped because ${__P(
currentThreads,0)} will return 0. At the same iteration, lines 4 through 6
(Continue reading)

Flavio Cysne | 15 Jun 2012 20:20
Picon

Re: JMeter threading model

Errata:

The first "If Controller" is executed and update a property named
"currentThreads" ...

2012/6/15 Flavio Cysne <flaviocysne <at> gmail.com>

> I have tested something in JMeter that could achieve a dynamic threading
> model and there was no need of new implementations.
>
> Basically, I have this structure:
>
> 1. Test Plan: { variables : [ {name: "maxThreads", value: 5}, {name:
> "currentThreads", value: 1} ] }
> 2. | - Thread Group: { threads: "${maxThreads}", rampup: 0, infinite: true
> }
> 3.     | - If Controller: { condition: "(${__threadNum()} ===
> ${maxThreads})" }
> 4.         | - Sampler /* I've used HTTP Request, but anyone would be used
> */
> 5.             | - RegEx Extractor: {refName: "threads", expression:
> "(.*)", model: "$1$", matchNo: "1", defaultValue: "0"}
> 6.             | - BeanShell Post-Processor: {language: "javascript",
> script: "var threads = vars.get(\"threads\");\nprops.put(\"currentThreads
> \",threads);"}
> 7.     | - If Controller: { condition: "(${__threadNum()} <
> ${maxThreads})" }
> 8.         | - If Controller: { condition: "(${__threadNum()} <= ${__P(
> currentThreads,0)})" }
> 9.             | - All other Samplers go here
(Continue reading)

Flavio Cysne | 17 Jun 2012 14:57
Picon

Re: JMeter threading model

Post entry in my blog:
http://flaviocysne.blogspot.com.br/2012/06/how-to-dynamically-increasedecrease.html

It's an unfinished work, but it's suffice to make what is proposed.
Comments and constructive criticism are welcome.

Hope it helps you.
Flávio Cysne

2012/6/15 Flavio Cysne <flaviocysne <at> gmail.com>

> Errata:
>
> The first "If Controller" is executed and update a property named
> "currentThreads" ...
>
>
> 2012/6/15 Flavio Cysne <flaviocysne <at> gmail.com>
>
>> I have tested something in JMeter that could achieve a dynamic threading
>> model and there was no need of new implementations.
>>
>> Basically, I have this structure:
>>
>> 1. Test Plan: { variables : [ {name: "maxThreads", value: 5}, {name:
>> "currentThreads", value: 1} ] }
>> 2. | - Thread Group: { threads: "${maxThreads}", rampup: 0, infinite:
>> true }
>> 3.     | - If Controller: { condition: "(${__threadNum()} ===
>> ${maxThreads})" }
(Continue reading)

sebb | 17 Jun 2012 21:06
Picon

Re: JMeter threading model

On 17 June 2012 13:57, Flavio Cysne <flaviocysne <at> gmail.com> wrote:
> Post entry in my blog:
> http://flaviocysne.blogspot.com.br/2012/06/how-to-dynamically-increasedecrease.html
>
> It's an unfinished work, but it's suffice to make what is proposed.
> Comments and constructive criticism are welcome.

Thanks!

The property used to control the thread count can also be set using
the BeanShell server, see:

http://jmeter.apache.org/usermanual/best-practices.html#beanshell_server

Or since properties are global, you could run the property update
sampler ("load controller") in a separate parallel thread group with
one thread.

However, JMeter will still have to start all the threads - even if
they are not actively sampling - which will take up memory.

It may not be too hard to change JMeter so it delays the thread
creation until required.
That may help in some use cases.

But it would probably be somewhat harder to allow threads to be
created later on demand.
And converting to an event-based model is definitely a lot of work.

> Hope it helps you.
(Continue reading)

Philippe Mouawad | 14 Jul 2012 15:36
Picon

Re: JMeter threading model

Hello,
Just a little mail to inform you that this feature has been implemented as
part of:

   - https://issues.apache.org/bugzilla/show_bug.cgi?id=53418

It is currently available in nightly build:

   - http://ci.apache.org/projects/jmeter/nightlies/

 <at> Kirk, it would be nice from you to look at this feature to see if it
answers your initial request and give us some feedback.

*Note that this feature may be subject to changes until next release, but
getting some feedback would be very useful.
*

Regards
Philippe M.

On Sun, Jun 17, 2012 at 9:06 PM, sebb <sebbaz <at> gmail.com> wrote:

> On 17 June 2012 13:57, Flavio Cysne <flaviocysne <at> gmail.com> wrote:
> > Post entry in my blog:
> >
> http://flaviocysne.blogspot.com.br/2012/06/how-to-dynamically-increasedecrease.html
> >
> > It's an unfinished work, but it's suffice to make what is proposed.
> > Comments and constructive criticism are welcome.
>
(Continue reading)

Kirk Pepperdine | 6 Aug 2012 12:40
Picon

Re: JMeter threading model

Hi Philippe,

Just got back from vacation going through email and I noticed that I missed this one. So I've now downloaded
the nightly build and loaded a test plan to check it out. Unfortunately the benchmark I decided to setup and
run against is borked. Well, I don't know if JMeter is the source of the problem my recent upgrade to
Mountain Lion (and what it did to Java). I'll retry on an older machine where the bench should still behave
as expected and then see how this new ThreadGroup functions. My guess is that it should be ok as it didn't
complain as it would have using a regular ThreadGroup. However, I would like to run a much much longer test
to make sure. I'll report back once this is done.

Regards,
Kirk

On 2012-07-14, at 3:36 PM, Philippe Mouawad <philippe.mouawad <at> gmail.com> wrote:

> Hello,
> Just a little mail to inform you that this feature has been implemented as part of:
> https://issues.apache.org/bugzilla/show_bug.cgi?id=53418
> It is currently available in nightly build:
> http://ci.apache.org/projects/jmeter/nightlies/
> 
>  <at> Kirk, it would be nice from you to look at this feature to see if it answers your initial request and give us
some feedback.
> 
> Note that this feature may be subject to changes until next release, but getting some feedback would be
very useful.
> 
> 
> Regards
> Philippe M.
(Continue reading)

Philippe Mouawad | 6 Aug 2012 15:25
Picon

Fwd: JMeter threading model

From kirk (bad subject due to vacation message :) ) :

Hi Phillippe,

The on demand thread group worked brilliantly for my short test. I'll
repeat with an 8 hour
run to give it an even better test and let you know how that worked.

-- Kirk

---------- Forwarded message ----------
From: *Kirk Pepperdine*
Date: Monday, August 6, 2012
Subject: JMeter threading model
To: Philippe Mouawad <philippe.mouawad <at> gmail.com>
Cc: JMeter Users List <user <at> jmeter.apache.org>

Hi Philippe,

Just got back from vacation going through email and I noticed that I missed
this one. So I've now downloaded the nightly build and loaded a test plan
to check it out. Unfortunately the benchmark I decided to setup and run
against is borked. Well, I don't know if JMeter is the source of the
problem my recent upgrade to Mountain Lion (and what it did to Java). I'll
retry on an older machine where the bench should still behave as expected
and then see how this new ThreadGroup functions. My guess is that it should
be ok as it didn't complain as it would have using a regular ThreadGroup.
However, I would like to run a much much longer test to make sure. I'll
report back once this is done.

(Continue reading)

Philippe Mouawad | 7 Aug 2012 09:54
Picon

Re: JMeter threading model

Hello Kirk,
Thank you very much for your feedback.
Waiting for the big test result.

Regards
Philippe

On Monday, August 6, 2012, Philippe Mouawad wrote:

> From kirk (bad subject due to vacation message :) ) :
>
>
>
> Hi Phillippe,
>
> The on demand thread group worked brilliantly for my short test. I'll repeat with an 8 hour
> run to give it an even better test and let you know how that worked.
>
> -- Kirk
>
>
> ---------- Forwarded message ----------
> From: *Kirk Pepperdine*
> Date: Monday, August 6, 2012
> Subject: JMeter threading model
> To: Philippe Mouawad <philippe.mouawad <at> gmail.com>
> Cc: JMeter Users List <user <at> jmeter.apache.org>
>
>
> Hi Philippe,
(Continue reading)

sebb | 31 Aug 2012 14:04
Picon

Re: JMeter threading model

Just to follow up - we have now implemented changes to the
TestCompiler and JavaSamplers to reduce the long-term memory storage
needed.

The memory requirement imposed by JMeter itself is now proportional to
the number of active threads; when a thread completes, its memory
should be released.

In conjunction with the Thread Group delayed thread creation option
and a long ramp-up, it's now possible to run a test with hundreds of
thousands of threads (possibly over a million), so long as the number
of concurrent (active) threads is kept to a reasonable number.

On 7 August 2012 08:54, Philippe Mouawad <philippe.mouawad <at> gmail.com> wrote:
> Hello Kirk,
> Thank you very much for your feedback.
> Waiting for the big test result.
>
> Regards
> Philippe
>
> On Monday, August 6, 2012, Philippe Mouawad wrote:
>
>> From kirk (bad subject due to vacation message :) ) :
>>
>>
>>
>> Hi Phillippe,
>>
>> The on demand thread group worked brilliantly for my short test. I'll repeat with an 8 hour
(Continue reading)

Shay Ginsbourg | 31 Aug 2012 17:12
Picon

Re: JMeter threading model

Well done!

Which JMeter version has all that implemented in?

On Fri, Aug 31, 2012 at 3:04 PM, sebb <sebbaz <at> gmail.com> wrote:
>
> Just to follow up - we have now implemented changes to the
> TestCompiler and JavaSamplers to reduce the long-term memory storage
> needed.
>
> The memory requirement imposed by JMeter itself is now proportional to
> the number of active threads; when a thread completes, its memory
> should be released.
>
> In conjunction with the Thread Group delayed thread creation option
> and a long ramp-up, it's now possible to run a test with hundreds of
> thousands of threads (possibly over a million), so long as the number
> of concurrent (active) threads is kept to a reasonable number.
>
> On 7 August 2012 08:54, Philippe Mouawad <philippe.mouawad <at> gmail.com> wrote:
> > Hello Kirk,
> > Thank you very much for your feedback.
> > Waiting for the big test result.
> >
> > Regards
> > Philippe
> >
> > On Monday, August 6, 2012, Philippe Mouawad wrote:
> >
> >> From kirk (bad subject due to vacation message :) ) :
(Continue reading)

sebb | 31 Aug 2012 17:52
Picon

Re: JMeter threading model

On 31 August 2012 16:12, Shay Ginsbourg <sginsbourg <at> gmail.com> wrote:
> Well done!
>
> Which JMeter version has all that implemented in?
>

The next version - which will likely be 2.8.

>
>
>
>
> On Fri, Aug 31, 2012 at 3:04 PM, sebb <sebbaz <at> gmail.com> wrote:
>>
>> Just to follow up - we have now implemented changes to the
>> TestCompiler and JavaSamplers to reduce the long-term memory storage
>> needed.
>>
>> The memory requirement imposed by JMeter itself is now proportional to
>> the number of active threads; when a thread completes, its memory
>> should be released.
>>
>> In conjunction with the Thread Group delayed thread creation option
>> and a long ramp-up, it's now possible to run a test with hundreds of
>> thousands of threads (possibly over a million), so long as the number
>> of concurrent (active) threads is kept to a reasonable number.
>>
>> On 7 August 2012 08:54, Philippe Mouawad <philippe.mouawad <at> gmail.com> wrote:
>> > Hello Kirk,
>> > Thank you very much for your feedback.
(Continue reading)

Shay Ginsbourg | 31 Aug 2012 18:08
Picon

Re: JMeter threading model

Looking forward!

Thanks!


www.ginsbourg.com | 054-6690915 |  Sent from my BlackBerry® smartphone from orange. 

-----Original Message-----
From: sebb <sebbaz <at> gmail.com>
Date: Fri, 31 Aug 2012 16:52:48 
To: JMeter Users List<user <at> jmeter.apache.org>
Reply-To: "JMeter Users List" <user <at> jmeter.apache.org>
Subject: Re: JMeter threading model

On 31 August 2012 16:12, Shay Ginsbourg <sginsbourg <at> gmail.com> wrote:
> Well done!
>
> Which JMeter version has all that implemented in?
>

The next version - which will likely be 2.8.

>
>
>
>
> On Fri, Aug 31, 2012 at 3:04 PM, sebb <sebbaz <at> gmail.com> wrote:
>>
>> Just to follow up - we have now implemented changes to the
>> TestCompiler and JavaSamplers to reduce the long-term memory storage
(Continue reading)

Kirk Pepperdine | 15 Jun 2012 07:22
Picon

Re: JMeter threading model

>> 
>> of servers (operators) is fixed and gate entry into the system. However,
>> I'm often simulating open systems which means I do not want rate of entry
>> into the system to be controlled by the performance of the system (rate of
>> exit).
> 
> What makes you think JMeter does this ?

math (queuing theory) and observation from profiling JMeter.

> 
> 
> 
>> and the quality of the results
> 
> I disagree with this assertion . We have been using JMeter for load testing
> all kind of applications Intranets / Large ECommerce Systems / Backoffice
> systems / , and quality of results is good provided you configure it
> properly.
> Particularly when using Remote  Testing. Lot of users in this mailing list
> use it and are satisfied (I think).

I'm happy to engage JMeter project developers in a constructive conversation if they are interested and I
shan't be offended if they choose to ignore me. What I don't feel is a the need to get into this type of a long
protracted discussion. As I've said, JMeter has a lot of admiral qualities which is why I use it and have
been introducing people how to use it for years. All I'm suggesting here are ways in which the tool can be
improved so that scales better and has less of an artificial influence on profiling results. For example,
I've tried to use it to drive a websocket app that was driving traffic to 1,000,000 simulated end users. We
tried using JMeter but the threading model simply prevented it so I took on the arduous task of custom
rolling an injector (not a route that I wanted to take). As any of the JMeter developers will tell you, load
(Continue reading)

Shmuel Krakower | 12 Jun 2012 18:08
Picon

Re: TCP Sampler Read Exception

Feel free to copy and paste here the complete error.

Shmuel.

On Tue, Jun 12, 2012 at 12:02 PM, sebb <sebbaz <at> gmail.com> wrote:

> On 12 June 2012 09:56, Shay Ginsbourg <sginsbourg <at> gmail.com> wrote:
> > Hi all,
> >
> > Can anyone suggest what does the following error mean ?
> >
> > After each TCP sampler, we receive "Response code: 500" and
> >
> > "Response message: org.apache.jmeter.protocol.tcp.sampler.ReadException:"
> >
> > Any help would be greatly appreciated.
>
> Most likely the read terminated unexpectedly:
>
>
> http://jmeter.apache.org/api/org/apache/jmeter/protocol/tcp/sampler/TCPClient.html#read%28java.io.InputStream%29
>
> > thanks,
> > Shay
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe <at> jmeter.apache.org
> For additional commands, e-mail: user-help <at> jmeter.apache.org
>
>
(Continue reading)


Gmane