Load shapes not removing users when user_classes changes

[tiadobatima]

Prerequisites

• I am using the latest version of Locust
• I am reporting a bug, not asking a question

Description

Hello there... it looks like load shapes aren't removing users as one would expect.
It looks like that if I want to change the user classes in each tick to simulate stages as described in the doc , we can only add users/classes to previous stages, not remove them.
Using a slighly modified version of the example in the doc, this is what I'm trying to run exactly:

• stage 0: exactly 4 UserA in the first 10 seconds (no UserB)
• stage 1: exactly 4 UserB, between seconds 10-20 (no UserA)
• stage 2: exactly 4 UserA + UserB (2 each), between seconds 20-30

Something like this:

    stages = [
        {"duration": 10, "users": 4, "spawn_rate": 5 "user_classes": [UserA]},
        {"duration": 10, "users": 4, "spawn_rate": 5, "user_classes": [UserB]},
        {"duration": 10, "users": 4, "spawn_rate": 5, "user_classes": [UserA, UserB]},
    ]

To make troubleshooting easier, I setup the users so that:

• UserA only makes calls to google.com
• UserB only makes calls to yahoo.com

We'll notice that calls to yahoo.com (UserB) are never made.

Here's the slack thread for more info:
https://locustio.slack.com/archives/C3NUJ61DJ/p1714504631777939

Command line

python -m locust -f src/xxxx/test.py --users 5 --run-time 20 --json --loglevel DEBUG --class-picker

Locustfile contents

class UserA(HttpUser):
    wait_time = constant(1)

    @task
    def call(self):
        self.client.get("http://www.google.com")

class UserB(HttpUser):
    wait_time = constant(1)

    @task
    def call(self):
        self.client.get("http://www.yahoo.com")

class MyShape(LoadTestShape):
    use_common_options = True

    stage_classes = [
        [UserA],
        [UserB],
        [UserA, UserB],
    ]

    def tick(self):
        total_run_time = 30
        users = 4
        spawn_rate = 5
        print(f"Total run time: {total_run_time}, Spawn rate: {spawn_rate}, Users: {users}")
        current_run_time = self.get_run_time()
        stages = len(self.stage_classes)
        stage_run_time = round(total_run_time/stages)
        current_stage = int(current_run_time/stage_run_time)
        stage_cut_off = stage_run_time + stage_run_time * (current_stage)

        if current_stage > stages - 1:
            return None

        print(f"tick: {current_run_time}, cutoff: {stage_cut_off}, current stage: {current_stage}, class: {self.stage_classes[current_stage]}")
        return (users, spawn_rate, self.stage_classes[current_stage])

Python version

3.11

Locust version

2.27.0

Operating system

Amazonlinux 2023

[tiadobatima]

I feel the problem could be somewhere in the _dispatcher() method, since code relies on the current user count (self._current_user_count) to figure our how many users to stop. If I understood it correctly, if self._current_user_count and self._target_user_count are the same, nothing happens and no users will be stopped?

https://github.com/locustio/locust/blob/master/locust/dispatch.py#L161-L192

[cyberw]

Yea, I think this has probably never worked. Basically a changed user distribution in the return value from tick cant stop users, and I think it would be hard to fix.

A workaround could be just stopping all the users between the stages:

class MyShape(LoadTestShape):
    previous_stage = 0
    ...
    def tick(self):
        ...
        if current_stage != self.previous_stage:
            self.previous_stage = current_stage
            return (0, 1000)
        ...

[tiadobatima]

Thank you so much for looking into this @cyberw !

Nice workaround... I did try a version of that but I was returning the classes as well. I didn't think of not returning the classes in the tick() 🤦
I also tried stopping users directly, but it didn't work either (and I still don't know why):

    if current_stage != self.previous_stage:
            stop_users = {u.__name__: users for u in self.stage_classes[self.previous_stage]}
            for k, v in self.runner.user_classes_count.items():
                print(f"before stop user count: {k} -> {v}")
            self.runner.stop_users(stop_users)

[cyberw]

Hmm.. yea I dont know why that doesnt work.

I should probably document this as a limitation or something.