if expression problem, how best to solve ?

[StefanSa]

hi there,

have this if expression

if (is_string(.observer.egress.zone) && is_string(.observer.ingress.zone)) {
     if (match_any(.observer.egress.zone, [r'^LAN', r'^DMZ', r'^VPN', r'^WiFi'])) && match_any(.observer.ingress.zone, [r'^WAN']) {
      .network.direction = "inbound"
     }
}

but sporadically get this error message.

2022-01-05T09:47:48.048737Z  WARN vector::internal_events::remap: Mapping failed with event. error="function call error for \"match_any\" at (943:987): expected \"string\", got \"null\"" internal_log_rate_secs=30
thread 'vector-worker' panicked at 'event will be set', src/transforms/remap.rs:234:48

The observer zone values were assigned as follows.

if exists(.sophos.xg.src_zone_type) {
.observer.ingress.zone = .sophos.xg.src_zone_type
}
if exists(.sophos.xg.dst_zone_type) {
.observer.egress.zone = .sophos.xg.dst_zone_type
}

There is already no "null" values from the syslogserver and is also backed up with this.

. = compact(., string: true, array: true, null: true, nullish: true)

What am i doing wrong here ?

[spencergilbert]

Are .sophos.xg.src_zone_type and .sophos.xg.dst_zone_type existing on every event? IIRC path's that don't exist are considered to contain a null value, so that may be where this is happening.

Regardless, I think there's something wrong happening in VRL here - so cc'ing @JeanMertz and @StephenWakely to see if there have an idea why we're getting a panick.

@StefanSa can you share your entire VRL source?

[StefanSa]

@spencergilbert Spencer, here the syslog message.

"device_name=\"SFW\" timestamp=\"2022-01-05T17:52:42+0100\" device_model=\"XG230\" device_serial_id=\"1234567890\" log_id=\"010101 600001\" log_type=\"Firewall\" log_component=\"Firewall Rule\" log_subtype=\"Allowed\" log_version=1 severity=\"Information\" fw_rule_id=\"60\" nat_rule_id=\"0\" fw_rule_typ e=\"USER\" ips_policy_id=17 ether_type=\"Unknown (0x0000)\" in_interface=\"ipsec0\" out_interface=\"Port1\" src_mac=\"C4:F7:D5:B5:47:F4\" dst_mac=\"7C:5A:1C:4D:13:81\" src_i p=\"172.17.35.106\" src_country=\"R1\" dst_ip=\"172.16.34.224\" dst_country=\"R1\" protocol=\"TCP\" src_port=56058 dst_port=10051 src_zone_type=\"VPN\" src_zone=\"VPN\" dst_ zone_type=\"LAN\" dst_zone=\"LAN\" con_event=\"Start\" con_id=\"263747456\" hb_status=\"No Heartbeat\" app_resolved_by=\"Signature\" app_is_cloud=\"FALSE\" qualifier=\"New\"  in_display_interface=\"ipsec0\" out_display_interface=\"Port1\""

if (match_any(.observer.egress.zone, [r'^LAN', r'^DMZ', r'^VPN', r'^WiFi'])) && match_any(.observer.ingress.zone, [r'^WAN'])
I have the following guess, if in the first match_any, for example WAN is not found, the if sequence terminates with null and the second match_any fails because of the null.

[spencergilbert]

Thanks! Can you share the full config/script you're using here as well?

[StefanSa]

Hi @spencergilbert Spencer, surely
vector.zip

[StefanSa]

@spencergilbert
any chance to solve this problem?

[StephenWakely]

There is a definite bug here. I have raised #10746 to track the issue. Vector should not boot at all with the script as it stands since it can't guarantee that observer.egress.zone, .observer.ingress.zone.

It would be nice if the if predicate:

if (is_string(.observer.egress.zone) && is_string(.observer.ingress.zone)) {

was sufficient to inform VRL that these values need to be strings within the if block. However, VRL is not quite advanced enough yet.

Now, I'm not convinced that this is the only bug that you have discovered - because that if should have been sufficient to prevent VRL from trying to run the match_all with parameters that are not strings. I cannot reproduce the error when that if statement is there.

Is it possible that the errors occur when that if clause is not there? I see in the config you posted that they are commented out. Is it possible that when you encounter the error you only uncomment the match_all line?

Clutching at straws here. I think we should fix #10746 and then see if that fixes all the problems before trying to reproduce further.

[StefanSa]

@StephenWakely
Hi Stephen,
first of all thanks for the fast and professional support here, which i already mentioned to Spencer. This is not always a matter of course. Can you please tell me exactly what i should test here?

#if (is_string(.observer.egress.zone) && is_string(.observer.ingress.zone)) {
#     if (match_any(.observer.egress.zone, [r'^LAN', r'^DMZ', r'^VPN', r'^WiFi'])) && match_any(.observer.ingress.zone, [r'^WAN']) {
#      .network.direction = "inbound"
#    }
#}

[StephenWakely]

What you are doing here is correct as far as I can see. Thus my question that is it possible that when you get these errors you neglected to uncomment the if (is_string line?

If not, (and I realise I may be clutching at straws here!) then something very strange is going on that we will need dig further into.

When we fix #10746, you will need to coerce the fields to string, like:

if (is_string(.observer.egress.zone) && is_string(.observer.ingress.zone)) {
     if (match_any(string!(.observer.egress.zone), [r'^LAN', r'^DMZ', r'^VPN', r'^WiFi'])) && match_any(string!(.observer.ingress.zone), [r'^WAN']) {
      .network.direction = "inbound"
    }
}

[StefanSa]

@StephenWakely
Stephen, it works without any problem.
Thanks for your help.

[StefanSa]

@StephenWakely
But a bit too early, after a few minutes there are now these warning messages.
However, it seems that most values are written.

022-01-07T12:35:50.274019Z  INFO vector: Vector has started. debug="false" version="0.19.0" arch="x86_64" build_id="da60b55 2021-12-28"
2022-01-07T12:35:50.274038Z  INFO vector::app: API is disabled, enable by setting `api.enabled` to `true` and use commands like `vector top`.
2022-01-07T12:36:06.862052Z  WARN vector::internal_events::remap: Mapping failed with event. error="function call error for \"match_any\" at (1202:1255): function call error for \"string\" at (1212:1243): expected \"string\", got \"null\"" internal_log_rate_secs=30
2022-01-07T12:36:35.935513Z  WARN vector::internal_events::remap: Internal log [Mapping failed with event.] is being rate limited.
2022-01-07T12:36:48.685740Z  WARN vector::internal_events::remap: Internal log [Mapping failed with event.] has been rate limited 1 times.
2022-01-07T12:36:48.685766Z  WARN vector::internal_events::remap: Mapping failed with event. error="function call error for \"match_any\" at (1202:1255): function call error for \"string\" at (1212:1243): expected \"string\", got \"null\"" internal_log_rate_secs=30