Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

'Content-Length' header not always returned when enumerating HttpContentHeaders #16162

Open
mellinoe opened this issue Jan 19, 2016 · 18 comments · Fixed by #102416
Open

'Content-Length' header not always returned when enumerating HttpContentHeaders #16162

mellinoe opened this issue Jan 19, 2016 · 18 comments · Fixed by #102416
Labels
area-System.Net.Http documentation Documentation bug or enhancement, does not impact product or test code
Milestone

Comments

@mellinoe
Copy link
Contributor

Duplicated from internal bug 122111. Creating this tracking issue here so that we can trace these through the markers in the code back to the original internal bug tracker discussions if necessary.

@karelz
Copy link
Member

karelz commented Dec 14, 2016

Next step: We need a repro (@davidsh you said you might have it - it is simple ~6 lines), then debug it and fix it.
Warning: It is complicated.

@mellinoe
Copy link
Contributor Author

I have no context on this issue; I just made a tracking issue because we had references to an old bug tracker in the code, and I was worried we would forget about it entirely. This might not even be a problem anymore.

@davidsh
Copy link
Contributor

davidsh commented Dec 15, 2016

This repros in .NET Framework (Desktop) and .NET Core. It is an old bug from .NET Framework and has been there since we first shipped HttpClient in .NET 4.5. Over time, this bug has become more annoying and impactful to developers.

[Fact]
public void ContentLength_CreateStringContent_HeaderPresentInToString()
{
    string data = "This is a test";
    var content = new StringContent(data);
    string headers = content.Headers.ToString();
    Console.WriteLine(headers);
    Assert.IsTrue(headers.Contains("Content-Length"));
}

[Fact]
public void ContentLength_CreateStringContentAndAccessPropertyHeader_HeaderPresentInToString()
{
    string data = "This is a test";
    var content = new StringContent(data);
    
    // Calling the 'getter' for the ContentLength property will reconcile the strings cache.
    Console.WriteLine("content.Headers.ContentLength={0}", content.Headers.ContentLength);
    string headers = content.Headers.ToString();

    Console.WriteLine(headers);

    Assert.IsTrue(headers.Contains("Content-Length"));
}

Test [2] always passes. This is good.
Test [1] should pass but fails because of this lazy initialization bug with ContentLength property.

@satano
Copy link
Member

satano commented Aug 11, 2018

@karelz Hello. Can I take this?

@satano
Copy link
Member

satano commented Jan 25, 2019

Info about how to fix this from @Tratcher during this PR.

A) There are some common HttpContent implementations that know their length up front such as ByteArrayContent, StringContent, FormUrlEncodedContent, etc.. These types could eagerly set ContentLength in their constructors rather than waiting for TryComputeLength to be called. This would fix header enumeration and ToString for those types. This wouldn't help dynamic content types like StreamContent, but it's not clear to me that it's important for ToString to include a lazily calculated header.

B) It's the transport's job to make the final determination on if the request should use Content-Length or Chunked. It performs that rationalization before serializing the request to the wire.
Proposal: Change TryComputeLength to a public API, call it directly from the transports, and set ContentLength manually before sending the request.

These changes could be taken independently or together. After these changes you may be able to remove the implicit ContentLength behavior.

@satano
Copy link
Member

satano commented Jan 25, 2019

I am going to do the first part - option A.

@pawepaw
Copy link

pawepaw commented Jul 29, 2019

any update on that? i'm still facing that issue. Without calling stringContent.Headers.ContentLength; i'm not getting content-length header.

@karelz
Copy link
Member

karelz commented Nov 7, 2019

Triage: Option A is worth doing - it should be simple.
Then close the issue -- skip part B until we have customers complaining, note: part B may be breaking for streaming scenarios, etc. We should document why it is the way it is then.

@satano looks like you're not working on it, so unassigning.

@msftgits msftgits transferred this issue from dotnet/corefx Jan 31, 2020
@msftgits msftgits added this to the Future milestone Jan 31, 2020
@heemskerkerik
Copy link

@karelz I'd like to pick up this issue.

@karelz
Copy link
Member

karelz commented Oct 22, 2020

@heemskerkerik sorry for delayed answer, if you are still interested, we'd be happy to help you out to make this happen. Thanks!

@heemskerkerik
Copy link

@karelz I'm still interested.

I've made some local changes, and I ran into an issue. The solution suggested was to add the Content-Length header in the constructor of StringContent and similar HttpContents. However, when you then use them as parts of a MultipartContent, the 'part' will then also have a Content-Length header.

Reading the RFCs, this does not seem to be wrong, per se, but it is obviously unnecessary.

An obvious solution is to adjust MultipartContent to explicitly set the Content-Length header to null before serializing all the parts to a stream, or when adding a part.

Does this sound reasonable?

picenka21 pushed a commit to picenka21/runtime that referenced this issue Feb 18, 2022
@michal-zatloukal
Copy link

Any update on this so far? response.Content.Headers still doesn't yield the Content-Length header.

@dotnet-policy-service dotnet-policy-service bot added the in-pr There is an active PR which will close this issue when it is merged label May 18, 2024
@MihaZupan MihaZupan modified the milestones: Future, 9.0.0 May 30, 2024
@MihaZupan
Copy link
Member

For reference, #102416 fixes this for built-in HttpContent types where the length is known (ByteArrayContent, ReadOnlyMemoryContent, and StreamContent when the stream is seekable).
Plus any types derived from them (StringContent, FormUrlEncodedContent).

If you have other custom HttpContent implementations, the behavior remains unchanged, and you'll have to set the header yourself / force the content to be buffered.

@MihaZupan MihaZupan removed the help wanted [up-for-grabs] Good issue for external contributors label Jun 4, 2024
@MihaZupan
Copy link
Member

See #102986 (comment) -- this particular change was reverted.

Any update on this so far? response.Content.Headers still doesn't yield the Content-Length header.

@michal-zatloukal the issue you are hitting is likely something else. If the response content doesn't have a content length, it's either because it was not sent by the server, or because it was removed (e.g. #42789).

@michal-zatloukal
Copy link

michal-zatloukal commented Jun 7, 2024

@MihaZupan It must have been set by the server because Content-Length is only not present in the enum response.Content.Headers (which is bug) but if I access it via response.Content.Headers.ContentLength.Value, I can retrieve the value.

This makes implementing forwarding HTTP requests pain, because the response.Content.Headers actually doesn't contain all headers.

See sample #102986 (comment)

@michal-zatloukal
Copy link

michal-zatloukal commented Jun 7, 2024

I was digging a bit deeper and made a minimal reproduction repository:
https://github.com/michal-zatloukal/content-length-bug-repro

I think now I understand the problem completely. The issue occurs when the remote HTTP server doesn't specify Content-Length header. It is not then present in response.Content.Headers. This is fine.

But the response.Content.Headers.ContentLength (https://learn.microsoft.com/en-us/dotnet/api/system.net.http.headers.httpcontentheaders.contentlength?view=net-8.0#definition) returns the calculated value. This is in contrast to the public documentation which incorrectly says:
Gets or sets the value of the Content-Length content header on an HTTP response.

It returns the calculated value not the value of Content-Length header. So either the public interface is incorrect or the behaviour is wrong.

@MihaZupan could you please re-open the issue as it wasn't fixed by the mentioned PR which was reverted?

@MihaZupan MihaZupan reopened this Jun 7, 2024
@MihaZupan MihaZupan modified the milestones: 9.0.0, Future Jun 7, 2024
@MihaZupan MihaZupan added documentation Documentation bug or enhancement, does not impact product or test code and removed bug in-pr There is an active PR which will close this issue when it is merged labels Jun 7, 2024
@MihaZupan
Copy link
Member

We've reverted that particular (already limited) change because of backward compatibility concerns.
It also wouldn't have applied to the response content.

Updating the documentation for the ContentLength property makes sense. E.g. something along the lines of

Gets or sets the value of the Content-Length content header on an HTTP response.
+If the header was never set, the value may be calculated on demand if the content was already buffered, or via `HttpContent.TryComputeLength`.

P.S. For scenarios where you're proxying requests, I'd recommend looking at YARP.

@michal-zatloukal
Copy link

@MihaZupan I agree with the documentation change and understand the backward compatability concerns.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area-System.Net.Http documentation Documentation bug or enhancement, does not impact product or test code
Projects
None yet
Development

Successfully merging a pull request may close this issue.

9 participants