feat(csharp): add PII redaction in tracing, by default

[birschick-bq]

Adds PII redaction in tracing, by default.

The new class ActivityWithPii wraps the System.Diagnostics.Activity object and replicates its interface. By default, the methods assume that tag values passed to the methods contain PII, unless explicitly marked otherwise (i.e., isPii = false).

The previous TraceActivity overloads and extension method that are use Activity have been marked as Obsolete. They have been replaced with overloads using the new ActivityWithPii class.

As marking the TraceActivity overloads and extension methods as Obsolete causes the build to fail, I've update the drivers in two commits. The first commit uses #pragma to ignore the warnings. The second commit, implements the change to use the new ActivityWithPii class.

The FileExporter and FileListener have been updated to reveal the original that have been redacted - with the assumption that writing to the local file system is controlled by an authorized user.

[birschick-bq]

@CurtHagenlocher - let me know if you have some time to review this or if is a low priority for you. Thanks.

[davidhcoe]

I think we can remove the changes that are under the Drivers folder since those are on the deprecation path. The changes are only needed in the new Foundry repo.

[davidhcoe]

I think we can remove the changes that are under the Drivers folder since those are on the deprecation path. The changes are only needed in the new Foundry repo.

I see. You mention in the description the build was breaking without that.

[birschick-bq]

I think we can remove the changes that are under the Drivers folder since those are on the deprecation path. The changes are only needed in the new Foundry repo.

I see. You mention in the description the build was breaking without that.

I could remove the Obsolete attributes, now that I've found all references. Then add back the Obsolete attribute once the other files are deprecated driver files and removed.

[davidhcoe]

Yes lets do that.

[birschick-bq]

Yes lets do that.

@davidhcoe - compromise, I had bracketed the obsolete method calls with #pragma warning disable CS0618 / #pragma warning restore CS0618. This is my original suggestion. Let me know if this reduces the impact on about-to-be deprecated/removed driver code. Thanks.

[davidhcoe]

I would prefer to not have any changes in the Drivers folder. This will update the dates of all of the files and gives the illusion they are still under active development.

[birschick-bq]

@davidhcoe

• Removed any change from the drivers.
• Refactored new overloads to be non-ambiguous.

[birschick-bq]

@CurtHagenlocher

Let me know if you have some time to review this PR.

[CurtHagenlocher]

Let me know if you have some time to review this PR.

I'm sorry for having taken so long to get to this, but I knew I'd need to be able to spend some time understanding the bigger picture. If I understand the goal of this change correctly, it's that it would let unredacted traces in a service environment be written to the file system on-demand for individual troubleshooting purposes, while using System.Diagnostics.ActivityListener to integrate with the telemetry system in our proprietary application in a way that allows conditional redaction of sensitive values. Is that right?

The main concern I have with the approach in the PR is that it only works for drivers implemented in .NET that haven't been AOT-compiled and are running in same CLR instance as the ActivityListener. Do you have any ideas on how this could be extended to e.g. Snowflake or FlightSQL or future drivers implemented in Go or Rust?

The other thing which feels a little unsettling is that it basically projects elements of our own internal privacy model onto the shared specification for (.NET-based) ADBC drivers, and in a way that limits the ability to (for instance) use an OTel-compatible exporter to get data that we consider sensitive but which another consumer might not.

Unfortunately, the Open Telemetry guidance around "sensitive values" seems to be to "don't log them at all", which isn't very helpful in an environment where the regulatory landscape varies so much between jurisdictions and over time.

The only other approach that immediately comes to mind is to encode the semantics of the value in its key name, but I'm still thinking about this.