notes.qmd

---
execute:
  eval: false
---


 Current mechanism

In short, the final plot is an animated plotly scatterplot.

In theory, there's nothing to stop it using an alternative mode of action (e.g. svg), but one benefit of plotly is that it nicely deals with the intermediate paths of patients. It is also available in both Python and R with minimal changes and has extensive compatability with other tools - e.g. Streamlit, Dash.


There are a couple of key steps to setting up the visualisation
1. Adding logging steps to the model
2. Swapping the use of resources for simpy stores *containing* resources
3. Creating an object that stores resources - a 'scenario' object - which then informs the number of resources displayed
4. Iterating through the logs to make a minute-by-minute picture of the position of every patient (or any desired interval)
5. Using Plotly to display these logs


## 1. Adding logging steps to the model

Five key classes of events need to be logged for every patient:
- arrival
- queue
- resource use start
- resource use end *(could possibly be removed)*
- depart

Simple improvements required include applying consistency to naming (e.g. arrival and departure, arrive and depart, not a mixture of the two)

At present, five to six things are recorded per log. 'Pathway' could potentially be removed.

This whole structure could be rewritten to be significantly less verbose. It is written like this at present because of the ease of transforming this structure of dictionary to a dataframe and the flexibility of the structure, but exploring alternatives like key:value pairs of event:time could be explored.


Currenly, the key logs take the following format

Arrival:
```
self.full_event_log.append({
    'patient': self.identifier,
    'pathway': 'Simplest',
    'event_type': 'arrival_departure',
    'event': 'arrival',
    'time': self.env.now
})
```
Queueing:
```
self.full_event_log.append({
    'patient': self.identifier,
    'pathway': 'Simplest',
    'event': 'treatment_wait_begins',
    'event_type': 'queue',
    'time': self.env.now
})
```

Resource Use Start:
```
self.full_event_log.append({
    'patient': self.identifier,
    'pathway': 'Simplest',
    'event': 'treatment_begins',
    'event_type': 'resource_use',
    'time': self.env.now,
    'resource_id': treatment_resource.id_attribute
})
```

Resource Use End:
```
self.full_event_log.append({
    'patient': self.identifier,
    'pathway': 'Simplest',
    'event': 'treatment_complete',
    'event_type': 'resource_use_end',
    'time': self.env.now,
    'resource_id': treatment_resource.id_attribute
})
```

Departure:
```
self.full_event_log.append({
    'patient': self.identifier,
    'pathway': 'Simplest',
    'event': 'depart',
    'event_type': 'arrival_departure',
    'time': self.env.now
})
```

## 2. Swapping the use of resources for simpy stores *containing* resources
When a resource is in use, we need to be able to show a single entity consistently hogging the same resource throughout the full time they are using it.

Simpy resources do not inherently have any ID attribute.
After exploring options like monkey patching the resource class, a better alternative seemed to be using a simpy store - which does have an ID - instead of a straight resource.

Without this ID attribute, the default logic used to move entities through the steps results in them visually behaving like a queue, which makes it hard to understand how long someone has been using a resource for and is visually confusing.

Fortunately the code changes required are minimal. We initialise the store, then use a loop to create as many resources within that store as required.

```
def init_resources(self):
    '''
    Init the number of resources
    and store in the arguments container object

    Resource list:
        1. Nurses/treatment bays (same thing in this model)

    '''
    self.args.treatment = simpy.Store(self.env)

    for i in range(self.args.n_cubicles_1):
        self.args.treatment.put(
            CustomResource(
                self.env,
                capacity=1,
                id_attribute = i+1)
            )
```

Use of the resource then becomes

```
# Seize a treatment resource when available
treatment_resource = yield self.args.treatment.get()
```

When the timeout has elapsed, we then use the following code.

```
# Resource is no longer in use, so put it back in
self.args.treatment.put(treatment_resource)
```

This has additional benefits of making it easier to monitor the use of individual resources.

One thing that has been noticed is that the resources seem to be cycled through in order. For example, if you have 4 resources and all are available, but the last resource to be in use was resource 2, resource 3 will be seized the next time someone requires a resource. This may not be entirely realistic, and code to 'shake up' the resources after use may be worth exploring.

## 3. Creating an object that stores resources - a 'scenario' object - which then informs the number of resources displayed
At present this part of the code expects a scenario object.
This could be changed to
- expect a dictionary instead
- work with either a scenario or dictionary object (maybe if the route of expanding TM's approach to simpy modelling into an opinionated framework)

If going with the first option, the scenario class used in TM's work routinely could be expanded to include a method to export the required data to a dictionary format.

```{python}
events_with_resources = event_position_df[event_position_df['resource'].notnull()].copy()
events_with_resources['resource_count'] = events_with_resources['resource'].apply(lambda x: getattr(scenario, x))

events_with_resources = events_with_resources.join(events_with_resources.apply(
    lambda r: pd.Series({'x_final': [r['x']-(10*(i+1)) for i in range(r['resource_count'])]}), axis=1).explode('x_final'),
    how='right')

fig.add_trace(go.Scatter(
    x=events_with_resources['x_final'].to_list(),
    # Place these slightly below the y position for each entity
    # that will be using the resource
    y=[i-10 for i in events_with_resources['y'].to_list()],
    mode="markers",
    # Define what the marker will look like
    marker=dict(
        color='LightSkyBlue',
        size=15),
    opacity=0.8,
    hoverinfo='none'
))
```

## 4. Iterating through the logs to make a minute-by-minute picture of the position of every patient (or any desired interval)
The function `reshape_for_animations()`

## 5. Using Plotly to display these logs

The function animate_activity_log currently takes 3 mandatory parameters:
- *full_patient_df*
- *event_position_df*
- *scenario*

*full_patient_df *is the output of the function **reshape_for_animations**


The graph is a plotly scatterplot.
The initial animated plot is created using plotly express, with additional static layers added afterwards.

Each individual is a scatter point. The actual points are fully transparent, and what we see is a text label - the emoji.

A list of any length of emojis is required. This will then be joined with a distinct patient table to provide a list of patients.


# Examples required

## Already created
- Simple pathway (units: minutes)
- Pathway with branching and optional steps (units: weeks)


## Not yet created - additional features possibly required
- Simple pathway (units: days, weeks)
- Resource numbers that change at different points of the day
- Prioritised queueing
- Shared resources
- Multiple resources required for a step (e.g. doctor + cubicle - how to display this?)
- Reneging
- Jockeying
- Balking



# Other comments

## Known areas for attention
- The code is not written in an object oriented manner.
- There's a bug in the wrapping code that results in queues building out in a diagonal manner (shifted 1 to the left) from the 3rd row onwards (2nd row counts to 11 instead of 10, and then subsequent rows correctly include 10 but start too far over)

## Required enhancements
- At present, the queue of users will continue to grow indefinitely until it leaves the boundary.

## Friction points
- Setting up the background image can be a fiddly process


## Other limitations
- By avoiding emojis that were released after v12.0 of the emoji standard (released in early 2019), we can ensure compatability with most major OSs. Windows 10 has not been updated past this point. However, due to the nature of emojis, we cannot absolutely ensure full compatability across all systems.



## Concerns

- Currently, logging can cope with ~5 minute snapshots for 5 days of logs in a system that has ~10-60 people in the system at any given point in time.
This results in a self-contained plot of ~20mb when exported (for comparison, a self-contained line chart with some additional rectangles is <20kb).
    - 5 days was chosen as a good limit for the streamlit teaching app as it offered a good balance between speed and minimized the risk of crashing across different choices of parameters.
        - If significantly too few resources are provided at a given step, the size of the animation dataframe quickly gets out of hand (as people aren't getting through the system so the number of people in the system at each snapshot is very large)
            - Working on a way of displaying queues after a threshold number of people is reached will help significantly

# Discussion with Tom

1.

- VIS (visual interactive simulation)

Sell it as being able to look at the simulation log visually

- Can be used to describe model logic to a client

- Can be used for validation (extreme value, model logic)
    - e.g. very long process times
    - e.g. very low number of resources and high service time



2.  LLM???? Future
Getting prompts to be generated



"Towards visualiation"

Need

Journal of simulation


Alison + Tom, + Dan + me