Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[PubSub to Datadog] Updating Datadog template to include full pubSub message by default #1579

Open
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

dtru-ddog
Copy link

Purpose:
Given customer feedback and to help improve parity with legacy logging structure updating default includePubsubMessage option to be true.

Copy link

google-cla bot commented May 20, 2024

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

@dtru-ddog dtru-ddog force-pushed the dan.trujillo/update-includepubsubmessage-default branch from 0d76602 to bd1dfe2 Compare May 20, 2024 13:51
Copy link
Collaborator

@ash-ddog ash-ddog left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thank you! Let's just make sure to communicate with existing users, as I believe restarting an existing Dataflow job will change behavior moving forward.

@dtru-ddog dtru-ddog force-pushed the dan.trujillo/update-includepubsubmessage-default branch from bd1dfe2 to 7d68385 Compare May 20, 2024 14:26
@dtru-ddog dtru-ddog force-pushed the dan.trujillo/update-includepubsubmessage-default branch from 7d68385 to a4804c8 Compare May 28, 2024 16:23
Copy link

codecov bot commented May 30, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 41.26%. Comparing base (a1f74a0) to head (aa65fe4).
Report is 98 commits behind head on main.

Additional details and impacted files
@@             Coverage Diff              @@
##               main    #1579      +/-   ##
============================================
- Coverage     44.79%   41.26%   -3.53%     
- Complexity      700     2884    +2184     
============================================
  Files           297      746     +449     
  Lines         15894    43633   +27739     
  Branches       1569     4645    +3076     
============================================
+ Hits           7119    18005   +10886     
- Misses         8245    24111   +15866     
- Partials        530     1517     +987     
Components Coverage Δ
spanner-templates 60.96% <85.00%> (+2.79%) ⬆️
spanner-import-export 64.47% <ø> (∅)
spanner-live-forward-migration 73.38% <ø> (ø)
spanner-live-reverse-replication 48.60% <ø> (+0.11%) ⬆️
spanner-bulk-migration 81.69% <85.00%> (+<0.01%) ⬆️
Files Coverage Δ
...ogle/cloud/teleport/templates/PubSubToDatadog.java 22.40% <100.00%> (ø)
...d/teleport/templates/common/DatadogConverters.java 81.25% <ø> (ø)

... and 466 files with indirect coverage changes

@@ -40,7 +40,7 @@ on [Metadata Annotations](https://github.com/GoogleCloudPlatform/DataflowTemplat
* **apiKey** : The Datadog API key. You must provide this value if the `apiKeySource` is set to `PLAINTEXT` or `KMS`. For more information, see API and Application Keys (https://docs.datadoghq.com/account_management/api-app-keys/) in the Datadog documentation.
* **batchCount** : The batch size for sending multiple events to Datadog. The default is `1` (no batching).
* **parallelism** : The maximum number of parallel requests. The default is `1` (no parallelism).
* **includePubsubMessage** : Whether to include the full Pub/Sub message in the payload. The default is `false` (only the data element is included in the payload).
* **includePubsubMessage** : Whether to include the full Pub/Sub message in the payload. The default is `true` (all elements, including data element, are included in the payload).
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We usually do not recommend changing the default value. What if we just update the doc to mention setting this to True is recommended?

@@ -65,7 +65,7 @@ variable "parallelism" {

variable "includePubsubMessage" {
type = bool
description = "Include full Pub/Sub message in the payload (true/false). Defaults to false (only data element is included in the payload)."
description = "Include full Pub/Sub message in the payload (true/false). Defaults to true (all elements, including data element, are included in the payload)."
default = null

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we update the default here as well?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants