You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Please vote on this issue by adding a 馃憤 reaction to the original issue to help the community and maintainers prioritize this request
If you are interested in working on this issue or have submitted a pull request, please leave a comment
Problem
Somewhere between the 0.34.0 release and 0.38.0, the vector_component_errors_total metrics appears to have disappeared(?).
I see the following in my logs:
May 17 08:07:27 host vector[1136]: 2024-05-17T08:07:27.872408Z WARN vector::internal_events::dnstap: Internal log [Recoverable error occurred while parsing dnstap data.] has been suppressed 3 times.
May 17 08:07:27 host vector[1136]: 2024-05-17T08:07:27.872542Z WARN vector::internal_events::dnstap: Recoverable error occurred while parsing dnstap data. error=Encountered error from TrustDns: unexpected end of input reached stage="processing" error_type="parser_failed" internal_log_rate_limit=true
May 17 08:07:27 host vector[1136]: 2024-05-17T08:07:27.873715Z WARN vector::internal_events::dnstap: Internal log [Recoverable error occurred while parsing dnstap data.] is being suppressed to avoid flooding.
May 17 08:13:20 host vector[1136]: 2024-05-17T08:13:20.391033Z WARN vector::internal_events::dnstap: Internal log [Recoverable error occurred while parsing dnstap data.] has been suppressed 3 times.
May 17 08:13:20 host vector[1136]: 2024-05-17T08:13:20.391127Z WARN vector::internal_events::dnstap: Recoverable error occurred while parsing dnstap data. error=Encountered error from TrustDns: more than one edns record present stage="processing" error_type="parser_failed" internal_log_rate_limit=true
May 17 08:13:20 host vector[1136]: 2024-05-17T08:13:20.391838Z WARN vector::internal_events::dnstap: Internal log [Recoverable error occurred while parsing dnstap data.] is being suppressed to avoid flooding.
May 17 09:15:11 host vector[1136]: 2024-05-17T09:15:11.058375Z WARN vector::internal_events::dnstap: Internal log [Recoverable error occurred while parsing dnstap data.] has been suppressed 3 times.
May 17 09:15:11 host vector[1136]: 2024-05-17T09:15:11.058462Z WARN vector::internal_events::dnstap: Recoverable error occurred while parsing dnstap data. error=Encountered error from TrustDns: unknown OpCode: 1 stage="processing" error_type="parser_failed" internal_log_rate_limit=true
May 17 09:15:11 host vector[1136]: 2024-05-17T09:15:11.059278Z WARN vector::internal_events::dnstap: Internal log [Recoverable error occurred while parsing dnstap data.] is being suppressed to avoid flooding.
May 17 09:26:12 host vector[1136]: 2024-05-17T09:26:12.854541Z WARN vector::internal_events::dnstap: Internal log [Recoverable error occurred while parsing dnstap data.] has been suppressed 3 times.
May 17 09:26:12 host vector[1136]: 2024-05-17T09:26:12.854659Z WARN vector::internal_events::dnstap: Recoverable error occurred while parsing dnstap data. error=Encountered error from TrustDns: unexpected end of input reached stage="processing" error_type="parser_failed" internal_log_rate_limit=true
May 17 09:26:12 host vector[1136]: 2024-05-17T09:26:12.855431Z WARN vector::internal_events::dnstap: Internal log [Recoverable error occurred while parsing dnstap data.] is being suppressed to avoid flooding.
May 17 09:41:14 host vector[1136]: 2024-05-17T09:41:14.241636Z WARN vector::internal_events::dnstap: Internal log [Recoverable error occurred while parsing dnstap data.] has been suppressed 3 times.
May 17 09:41:14 host vector[1136]: 2024-05-17T09:41:14.241733Z WARN vector::internal_events::dnstap: Recoverable error occurred while parsing dnstap data. error=Encountered error from TrustDns: unknown OpCode: 1 stage="processing" error_type="parser_failed" internal_log_rate_limit=true
May 17 09:41:14 host vector[1136]: 2024-05-17T09:41:14.241979Z WARN vector::internal_events::dnstap: Internal log [Recoverable error occurred while parsing dnstap data.] is being suppressed to avoid flooding.
May 17 09:45:30 host vector[1136]: 2024-05-17T09:45:30.053433Z WARN vector::internal_events::dnstap: Internal log [Recoverable error occurred while parsing dnstap data.] has been suppressed 3 times.
May 17 09:45:30 host vector[1136]: 2024-05-17T09:45:30.053588Z WARN vector::internal_events::dnstap: Recoverable error occurred while parsing dnstap data. error=Encountered error from TrustDns: unexpected end of input reached stage="processing" error_type="parser_failed" internal_log_rate_limit=true
May 17 09:45:30 host vector[1136]: 2024-05-17T09:45:30.053970Z WARN vector::internal_events::dnstap: Internal log [Recoverable error occurred while parsing dnstap data.] is being suppressed to avoid flooding.
May 17 09:45:40 host vector[1136]: 2024-05-17T09:45:40.511702Z WARN vector::internal_events::dnstap: Internal log [Recoverable error occurred while parsing dnstap data.] has been suppressed 3 times.
May 17 09:45:40 host vector[1136]: 2024-05-17T09:45:40.511801Z WARN vector::internal_events::dnstap: Recoverable error occurred while parsing dnstap data. error=Encountered error from TrustDns: unknown OpCode: 1 stage="processing" error_type="parser_failed" internal_log_rate_limit=true
May 17 09:45:40 host vector[1136]: 2024-05-17T09:45:40.512078Z WARN vector::internal_events::dnstap: Internal log [Recoverable error occurred while parsing dnstap data.] is being suppressed to avoid flooding.
May 17 09:52:40 host vector[1136]: 2024-05-17T09:52:40.936106Z WARN vector::internal_events::dnstap: Internal log [Recoverable error occurred while parsing dnstap data.] has been suppressed 3 times.
May 17 09:52:40 host vector[1136]: 2024-05-17T09:52:40.936213Z WARN vector::internal_events::dnstap: Recoverable error occurred while parsing dnstap data. error=Encountered error from TrustDns: unknown OpCode: 1 stage="processing" error_type="parser_failed" internal_log_rate_limit=true
May 17 10:12:43 host vector[1136]: 2024-05-17T10:12:43.021913Z WARN vector::internal_events::dnstap: Recoverable error occurred while parsing dnstap data. error=Encountered error from TrustDns: unknown OpCode: 1 stage="processing" error_type="parser_failed" internal_log_rate_limit=true
May 17 10:12:43 host vector[1136]: 2024-05-17T10:12:43.022860Z WARN vector::internal_events::dnstap: Internal log [Recoverable error occurred while parsing dnstap data.] is being suppressed to avoid flooding.
May 17 10:15:50 host vector[1136]: 2024-05-17T10:15:50.614936Z WARN vector::internal_events::dnstap: Internal log [Recoverable error occurred while parsing dnstap data.] has been suppressed 3 times.
May 17 10:15:50 host vector[1136]: 2024-05-17T10:15:50.617455Z WARN vector::internal_events::dnstap: Recoverable error occurred while parsing dnstap data. error=Encountered error from TrustDns: unknown OpCode: 1 stage="processing" error_type="parser_failed" internal_log_rate_limit=true
May 17 10:15:50 host vector[1136]: 2024-05-17T10:15:50.617852Z WARN vector::internal_events::dnstap: Internal log [Recoverable error occurred while parsing dnstap data.] is being suppressed to avoid flooding.
May 17 10:41:54 host vector[1136]: 2024-05-17T10:41:54.338652Z WARN vector::internal_events::dnstap: Internal log [Recoverable error occurred while parsing dnstap data.] has been suppressed 3 times.
May 17 10:41:54 host vector[1136]: 2024-05-17T10:41:54.338730Z WARN vector::internal_events::dnstap: Recoverable error occurred while parsing dnstap data. error=Encountered error from TrustDns: unknown OpCode: 1 stage="processing" error_type="parser_failed" internal_log_rate_limit=true
May 17 10:41:55 host vector[1136]: 2024-05-17T10:41:55.339354Z WARN vector::internal_events::dnstap: Internal log [Recoverable error occurred while parsing dnstap data.] is being suppressed to avoid flooding.
May 17 11:01:42 host vector[1136]: 2024-05-17T11:01:42.415285Z WARN vector::internal_events::dnstap: Internal log [Recoverable error occurred while parsing dnstap data.] has been suppressed 3 times.
May 17 11:01:42 host vector[1136]: 2024-05-17T11:01:42.415383Z WARN vector::internal_events::dnstap: Recoverable error occurred while parsing dnstap data. error=Encountered error from TrustDns: more than one edns record present stage="processing" error_type="parser_failed" internal_log_rate_limit=true
May 17 11:01:42 host vector[1136]: 2024-05-17T11:01:42.415558Z WARN vector::internal_events::dnstap: Internal log [Recoverable error occurred while parsing dnstap data.] is being suppressed to avoid flooding.
May 17 11:47:49 host vector[1136]: 2024-05-17T11:47:49.058416Z WARN vector::internal_events::dnstap: Internal log [Recoverable error occurred while parsing dnstap data.] has been suppressed 3 times.
May 17 11:47:49 host vector[1136]: 2024-05-17T11:47:49.058546Z WARN vector::internal_events::dnstap: Recoverable error occurred while parsing dnstap data. error=Encountered error from TrustDns: unrecognized label code: 1101001 stage="processing" error_type="parser_failed" internal_log_rate_limit=true
May 17 11:48:26 host vector[1136]: 2024-05-17T11:48:26.789594Z WARN vector::internal_events::dnstap: Recoverable error occurred while parsing dnstap data. error=Encountered error from TrustDns: unrecognized label code: 1110010 stage="processing" error_type="parser_failed" internal_log_rate_limit=true
May 17 11:48:26 host vector[1136]: 2024-05-17T11:48:26.797605Z WARN vector::internal_events::dnstap: Internal log [Recoverable error occurred while parsing dnstap data.] is being suppressed to avoid flooding.
May 17 12:14:47 host vector[1136]: 2024-05-17T12:14:47.779896Z WARN vector::internal_events::dnstap: Internal log [Recoverable error occurred while parsing dnstap data.] has been suppressed 1 times.
May 17 12:14:47 host vector[1136]: 2024-05-17T12:14:47.780053Z WARN vector::internal_events::dnstap: Recoverable error occurred while parsing dnstap data. error=Encountered error from TrustDns: unknown OpCode: 1 stage="processing" error_type="parser_failed" internal_log_rate_limit=true
May 17 12:14:47 host vector[1136]: 2024-05-17T12:14:47.789105Z WARN vector::internal_events::dnstap: Internal log [Recoverable error occurred while parsing dnstap data.] is being suppressed to avoid flooding.
May 17 12:28:28 host vector[1136]: 2024-05-17T12:28:28.604621Z WARN vector::internal_events::dnstap: Internal log [Recoverable error occurred while parsing dnstap data.] has been suppressed 3 times.
May 17 12:28:28 host vector[1136]: 2024-05-17T12:28:28.604796Z WARN vector::internal_events::dnstap: Recoverable error occurred while parsing dnstap data. error=Encountered error from TrustDns: unrecognized label code: 1100100 stage="processing" error_type="parser_failed" internal_log_rate_limit=true
May 17 12:28:28 host vector[1136]: 2024-05-17T12:28:28.638766Z WARN vector::internal_events::dnstap: Internal log [Recoverable error occurred while parsing dnstap data.] is being suppressed to avoid flooding.
May 17 14:06:45 host vector[1136]: 2024-05-17T14:06:45.992122Z WARN vector::internal_events::dnstap: Internal log [Recoverable error occurred while parsing dnstap data.] has been suppressed 35 times.
May 17 14:06:45 host vector[1136]: 2024-05-17T14:06:45.992290Z WARN vector::internal_events::dnstap: Recoverable error occurred while parsing dnstap data. error=Encountered error from TrustDns: unrecognized label code: 1101111 stage="processing" error_type="parser_failed" internal_log_rate_limit=true
May 17 14:06:46 host vector[1136]: 2024-05-17T14:06:46.135579Z WARN vector::internal_events::dnstap: Internal log [Recoverable error occurred while parsing dnstap data.] is being suppressed to avoid flooding.
May 17 14:26:09 host vector[1136]: 2024-05-17T14:26:09.865810Z WARN vector::internal_events::dnstap: Internal log [Recoverable error occurred while parsing dnstap data.] has been suppressed 11 times.
May 17 14:26:09 host vector[1136]: 2024-05-17T14:26:09.865914Z WARN vector::internal_events::dnstap: Recoverable error occurred while parsing dnstap data. error=Encountered error from TrustDns: more than one edns record present stage="processing" error_type="parser_failed" internal_log_rate_limit=true
May 17 14:26:09 host vector[1136]: 2024-05-17T14:26:09.866355Z WARN vector::internal_events::dnstap: Internal log [Recoverable error occurred while parsing dnstap data.] is being suppressed to avoid flooding.
( These errors might be of interest to @esensar ).
However, if I go to Grafana and explore under vector_ - I do not see any metric names that have error. I did have a dashboard with vector_component_errors_total that did have HTTP error metrics at version 0.34.0, but is now empty.
No output for error on the local Vector Prometheus endpoint with the above log:
Those logs you shared are actually just warnings rather than errors which is why they don't increment component_errors_total. Note that you will only see component_errors_total if there an error occurred (that is, it isn't published if there were no errors).
I think that resolves this question, but feel free to reopen if not.
A note for the community
Problem
Somewhere between the 0.34.0 release and 0.38.0, the
vector_component_errors_total
metrics appears to have disappeared(?).I see the following in my logs:
( These errors might be of interest to @esensar ).
However, if I go to Grafana and explore under
vector_
- I do not see any metric names that haveerror
. I did have a dashboard withvector_component_errors_total
that did have HTTP error metrics at version0.34.0
, but is now empty.No output for
error
on the local Vector Prometheus endpoint with the above log:Configuration
No response
Version
0.38.0
Debug Output
No response
Example Data
No response
Additional Context
No response
References
No response
The text was updated successfully, but these errors were encountered: