FabricObserver 3.2.6

FabricObserver 3.2.6, ClusterObserver 2.2.3 - SFPkgs with Microsoft-signed binaries. Microsoft-signed Nupkgs will be located in the nuget.org gallery.

Note: You can deploy FO/CO to your cluster directly by clicking on the Deploy to Azure button. You will need to specify the path to the related sfpkg, located in the Assets section (choose the type (SelfContained or FrameworkDependent, right-click the asset name, Copy. Then, paste into the package field in the ARM portal UI).

FabricObserver Changes

This release contains a bug fix for concurrent processing in AppObserver. Note: If you do not set AppObserverEnableConcurrentMonitoring to true, then you don't need to upgrade from 3.2.5 right away. Else, you should.

• Bug Fix: Issue #266
• Added new telemetry data types (SystemServiceTelemetryData and ContainerTelemetryData).

ClusterObserver Changes:

• Added support for FabricObserver's new TelemetryDataBase types, SystemServiceTelemetryData and ContainerTelemetryData.

FabricObserver 3.2.5

FabricObserver 3.2.5, ClusterObserver 2.2.2 - SFPkgs with Microsoft-signed binaries. Microsoft-signed Nupkgs will be located in the nuget.org gallery.

Note: You can deploy FO/CO to your cluster directly by clicking on the Deploy to Azure button. You will need to specify the path to the related sfpkg, located in the Assets section (choose the type (SelfContained or FrameworkDependent, right-click the asset name, Copy. Then, paste into the package field in the ARM portal UI).

Update: This release introduced a regression in AppObserver's concurrent processing impl. This is fixed in 3.2.6, which has been released. Note that you don't need to rush to upgrade if you don't enable parallel processing in AppObserver.

FabricObserver Changes

• FabricObserver now ships only as a .NET 6 application and targets SF Runtime versions 9 and higher. It will not run on SF runtime versions < 9. If you require that, then you will have to rebuild the product yourself and target the correct SDK. Or, just keep using 3.2.4.831 in that case. That said, you really should upgrade to SF 9.x at your earliest convenience.

• New telemetry data types: NodeTelemetryData, ServiceTelemetryData, DiskTelemetryData. These types hold only the relevant information for the target entity. For backwards compatibility, TelemetryData type still exists. This decreases the number of irrelevant properties that show up in data events for entities like Node and Disk.

• NodeObserver can be configured to emit Node snapshots (only useful if you employ ApplicationInsights/LogAnalytics/ETW).

• Bug fixes, performance and code improvements.

ClusterObserver Changes:

• ClusterObserver now ships only as a .NET 6 application and targets SF Runtime versions 9 and higher. It will not run on SF runtime versions < 9.
• Several core improvements and better support for emitting useful health information for any SF entity, regardless of health event origin (e.g., from a service like FabricObserver).
• Support for FabricObserver's updated telemetry data types.
• Bug fixes.
• It is recommended that you update to this version if you also deploy FabricObserver 3.2.5.

FabricObserver 3.2.4.960

FabricObserver 3.2.4.960 - SFPkgs with Microsoft-signed binaries. Microsoft-signed Nupkgs will be located in the nuget.org gallery.

NOTE: This release requires SF Runtime versions 9.0 and higher and targets .NET 6.

FabricObserver Changes

• Modifications to how AppObserver monitors RG memory limits for service processes (and their children): If you specify both MemoryInMB and MemoryInMBLimit, FO will always choose the absolute value specified in the MemoryInMBLimit setting for the code package. This feature enables you to have FO alert you when one of your services is approaching (d)OOM (Windows-only today. Linux support is coming). You can configure when FO should warn you (80% of the way there? 90%? Whatever makes your heart sing).

• Modifications to how AppObserver detects and monitors Guest Executable services and service helper code packages.

• Protection against over-calling netstat for TCP port monitoring. Windows does not offer a publicly documented API for getting BOUND state connection info (it’s simple to do, but not in a sanctioned way, therefore not in a Microsoft-owned public Github repo…). As such, FO must rely on making external calls to netstat (via createprocess), which can be relatively expensive both internally (what netstat does…) and externally (Windows Defender stalking FO == CPU abuse). That said, FO employs port data caching and does not abuse netstat.

• Bug fix: Guest Executable service data contains the usual information - Service name, partition id, replica id, etc.

• Bug fix: Service helper code package data contains related information about the primary service to which it belongs - Service name, partition id, replica id, etc. This is so you can easily correlate a helper code package to its "owning" service entity. Partition and replica don't really make sense for a helper code package itself, but they are important pieces of information about the related service instance.

• Removed support for ApplicationInsights InstrumentationKey usage. Instead, you must use the full Connection String. This is because ApplicationInsights has deprecated support for supplying only an InstrumentationKey for authentication/identification to its service. You will see this change reflected in FO's Settings.xml.

• Performance improvements.

• Updated nuget package dependencies to latest versions, where possible. Note: If you build plugins and do not use the FO Extensibility nuget package, you must update your Microsoft.Extensions.DependencyInjection.Abstractions reference to version 7.0.0.

FabricObserver 3.2.4.831

FabricObserver 3.2.4.831 - SFPkgs with Microsoft-signed binaries. Microsoft-signed Nupkgs will be located in the nuget.org gallery.

IMPORTANT: .NET Core 3.1 has reached its EOL and is no longer supported. If you are running SF Version 9 or higher (and you should be by now..), please use the .NET 6 release.

NOTE: This release requires SF Runtime versions 8.0 and higher and targets .NET Core 3.1.

FabricObserver Changes

Note: this is the last version of FabricObserver that will support .NET Core 3.1 and Service Fabric runtime 8.x. Going forward, there will only be one release and it will target .NET 6 and Service Fabric runtime versions 9 and higher.

• Modifications to how AppObserver monitors RG memory limits for service processes (and their children): If you specify both MemoryInMB and MemoryInMBLimit, FO will always choose the absolute value specified in the MemoryInMBLimit setting for the code package. This feature enables you to have FO alert you when one of your services is approaching (d)OOM (Windows-only today. Linux support is coming). You can configure when FO should warn you (80% of the way there? 90%? Whatever makes your heart sing).

• Modifications to how AppObserver detects and monitors Guest Executable services and service helper code packages.

• Protection against over-calling netstat for TCP port monitoring. Windows does not offer a publicly documented API for getting BOUND state connection info (it’s simple to do, but not in a sanctioned way, therefore not in a Microsoft-owned public Github repo…). As such, FO must rely on making external calls to netstat (via createprocess), which can be relatively expensive both internally (what netstat does…) and externally (Windows Defender stalking FO == CPU abuse). That said, FO employs port data caching and does not abuse netstat.

• Bug fix: Guest Executable service data contains the usual information - Service name, partition id, replica id, etc.

• Bug fix: Service helper code package data contains related information about the primary service to which it belongs - Service name, partition id, replica id, etc. This is so you can easily correlate a helper code package to its "owning" service entity. Partition and replica don't really make sense for a helper code package itself, but they are important pieces of information about the related service instance.

• Removed support for ApplicationInsights InstrumentationKey usage. Instead, you must use the full Connection String. This is because ApplicationInsights has deprecated support for supplying only an InstrumentationKey for authentication/identification to its service. You will see this change reflected in FO's Settings.xml.

• Performance improvements.

• Updated nuget package dependencies to latest versions, where possible.

FabricObserver 3.2.3.960

FabricObserver 3.2.3.960 - SFPkgs with Microsoft-signed binaries. Microsoft-signed Nupkgs will be located in the nuget.org gallery.

NOTE: This release requires SF Runtime versions 9.0 and higher and targets .NET 6.

FabricObserver Changes

• Bug Fix (OSInfoProvider - Windows implementation). This version fixes a bug with the Logger implementation in WindowsOSInfoProvider that leads to a NullReferenceException.
• Updated Windows TCP port detection impl to account for BOUND state connections (netstat is used to get port data in this release).
• Updated OSInfoProvider.TupleGetDynamicPortRange impl.

FabricObserver 3.2.3.831

FabricObserver 3.2.3.831 - SFPkgs with Microsoft-signed binaries. Microsoft-signed Nupkgs will be located in the nuget.org gallery.

NOTE: This release requires SF Runtime versions 8.0 and higher and targets .NET Core 3.1.

FabricObserver Changes

• Bug Fix (OSInfoProvider - Windows implementation). This version fixes a bug with the Logger implementation in WindowsOSInfoProvider that leads to a NullReferenceException.
• Updated Windows TCP port detection impl to account for BOUND state connections (netstat is used to get port data in this release).
• Updated OSInfoProvider.TupleGetDynamicPortRange impl.

FabricObserver 3.2.2.831

FabricObserver 3.2.2.831, ClusterObserver 2.2.1.831 - SFPkgs with Microsoft-signed binaries. Microsoft-signed Nupkgs will be located in the nuget.org gallery.

NOTE: This release requires SF Runtime versions 8.0 and higher and targets .NET Core 3.1.

FabricObserver Changes

• AppObserver will now detect and monitor Guest Executable services.
• AppObserver can now be configured to monitor RG Memory Limits and Warn (only) when 90% of said limit has been reached. This is configurable in AppObserver configuration (JSON) so you can override the default value per application. This includes all service CodePackages with RG Memory limits specified (MemoryInMb or MemoryInMbLimit). This is Windows-only for now.
• AppObserver now monitors all code packages (e.g., helper binaries) specified in ServiceManifest.
• AppObserver now monitors process Private Bytes (Windows-only, MB and Percent).
• AppObserver can now dump service processes in Warning state.

See the Using documentation for samples and more info.

ClusterObserver Changes

• Bug fixes

FabricObserver 3.2.2.960

FabricObserver 3.2.2.960, ClusterObserver 2.2.1.960 - SFPkgs with Microsoft-signed binaries. Microsoft-signed Nupkgs will be located in the nuget.org gallery.

NOTE: This release requires SF Runtime versions 9.0 and higher and targets .NET 6.

FabricObserver Changes

• AppObserver will now detect and monitor Guest Executable services.
• AppObserver can now be configured to monitor RG Memory Limits and Warn (only) when 90% of said limit has been reached. This is configurable in AppObserver configuration (JSON) so you can override the default value per application. This includes all service CodePackages with RG Memory limits specified (MemoryInMb or MemoryInMbLimit). This is Windows-only for now.
• AppObserver now monitors all code packages (e.g., helper binaries) specified in ServiceManifest.
• AppObserver now monitors process Private Bytes (Windows-only, MB and Percent).
• AppObserver can now dump service processes in Warning state.

See the Using documentation for samples and more info.

ClusterObserver Changes

• Bug fixes

FabricObserver 3.2.1.960

FabricObserver 3.2.1.960 - SFPkgs with Microsoft-signed binaries. Microsoft-signed Nupkgs will be located in the nuget.org gallery.

NOTE: This release requires SF Runtime versions 9.0 and higher and targets .NET 6.

FabricObserver Changes (NOTE: Breaking ETW changes - Kusto queries will have to be modified.)

• AppObserver: Updated ChildProcessInfo to include ProcessId of child process.
• Bug fix: FO will no longer put itself into Warning with an infinite TTL health event when some observer does not complete within user-specified max run time.
• Bug fix: AppObserver - Same named child processes will now be processed correctly.
• Bug fix: FabricSystemObserver regression in reporting for some system services has been resolved.
• DiskObserver: TelemetryData Property value is now "[drive] [Metric name]", e.g., on Windows, "C: Disk Space Usage (Percent)".
• Performance improvements.
• New ETW payload format: Json-serialized instances of *TelemetryData or anonymous types. This breaks existing KQL queries. See new ETW documentation for info.

FabricObserver 3.2.1.831

FabricObserver 3.2.1.831 - SFPkgs with Microsoft-signed binaries. Microsoft-signed Nupkgs will be located in the nuget.org gallery.

NOTE: This release requires SF Runtime versions 8.0 and higher and targets .NET Core 3.1.

FabricObserver Changes (NOTE: Breaking ETW changes - Kusto queries will have to be modified.)

• AppObserver: Updated ChildProcessInfo to include ProcessId of child process.
• Bug fix: FO will no longer put itself into Warning with an infinite TTL health event when some observer does not complete within user-specified max run time.
• Bug fix: AppObserver - Same named child processes will now be processed correctly.
• Bug fix: FabricSystemObserver regression in reporting for some system services has been resolved.
• DiskObserver: TelemetryData Property value is now "[drive] [Metric name]", e.g., on Windows, "C: Disk Space Usage (Percent)".
• Performance improvements.
• New ETW payload format: Json-serialized instances of *TelemetryData or anonymous types. This breaks existing KQL queries. See new ETW documentation for info.