Add support for streaming on requests (and add better handling for streaming in responses)

[trumpetinc]

Related issue - my apologies if this is a double post - I wasn't sure how to make sure the discussion was being viewed by the appropriate people: #1243

Problem

Feign currently uses a byte[] when processing the request body. This does not work for transfers of large request bodies (for example, uploading large files).

Requested solution

Make request bodies be designed around streams.

Status/Findings

I just finished up creating a bunch of shims to make streaming requests work in Feign, and would like to see if ya'll would be open to having a discussion about how to make this part of the main library. This works, but it is ugly.

Doing this properly will require a breaking change, so I am hesitant to create a pull request without some good discussion about the best way to do it, where we can and can't break API, etc...

Possible Approach

Based on the workaround implementation I created, I believe that the general approach will be as follows:

First, the existing Response.Body class is perfectly suitable for handing both Request and Responses. It actually looks like Response.Body was designed with this in mind. I think that refactoring this into a standalone Body interface, with concrete implementations for ByteArrayBody, InputStreamBody, NoContentBody, and possibly a FileContentBody. I also suggest removing calls specific to encoding, etc... from this class. I would like to get rid of null checks on the body parameter and just use a no-op NoContentBody if possible.

Next, there will be some surgical changes to Client.Default (for requests, we need to write a stream instead of a byte[])

There are some questions about calling close() (or not calling close() ) at the end of the response.

Another big impact area is going to be in logger implementations (which need to either not do body logging on streamed bodies, or need to use a different approach that monitors the stream but does not actually read the stream as part of the logging call).

And I think there should be some discussion about supporting InputStream as a body parameter type, and possibly OutputStream as a client method response parameter type. Maybe these could be handled with specialized encoder/decoder implementations, but it's worth discussion about whether these should be part of the core library. Might also discuss a File type as a body parameter type (this would allow setting content-length header and make the body retryable, which would worthwhile.

Finally, all of my work has been on the synchronous implementations - so evaluation with respect to Async will be needed.

Conclusion

All in all, while the design discussion will be important, I am confident that it is possible to make this change without massive amounts of surgery. And I think this would be a huge enhancement to Feign. This is a huge gap with Jax-RS that would be good to close.

Is there interest in pursuing this? I'm happy to do the grunt work on the pull request, but I think this is a big enough breaking change that there should be discussion to get the design right before I start slinging code.

[kdavisk6]

Thanks for taking the time to write this up. You are on the right track by understanding that our preference is to keep the core Feign objects simple and rely on specialized Encoder and Decoder implementations to deal with these sorts of scenarios. We've learned that trying to include this type of capabilities into core Feign result in best intentions gone wrong.

With that said, including OutputStream and InputStream specific implementations would be a welcome addition. Please feel free to submit a Pull Request with your ideas for our review.

[trumpetinc]

ok - I will put together a PR over the next week or so, then we can have what I'm sure will be a lively discussion about some of the tradeoffs.

[rickgongtds]

It would be great if Feign could provide an elegant way to support streaming large files.

Our organization has hundreds of applications using Feign clients to upload files to a central file service. While most files are relatively small (less than 20MB), there are occasional cases where files can reach sizes up to 2GB. Unfortunately, this causes our services to experience frequent OutOfMemory (OOM) errors due to Feign loading the entire file into memory.

To address this issue, I had to resort to a temporary and admittedly ugly hack. Specifically, I recompiled the FeignInvocationHandler class to handle large file uploads with a custom workaround. Here's an excerpt of the adjusted invoke() method:

@Override
public Object invoke(Object proxy, Method method, Object[] args) throws Throwable {
    // ...
    if (method.getDeclaringClass() == MyBigFileFeignClient.class &&
            method.getName().equals("uploadBigFile")) {
        return MyUglyTempWorkAround.streamingLargeFile(method, args);
    }

    return dispatch.get(method).invoke(args);
}

The class MyUglyTempWorkAround will use OkHttpClient directly to stream large file, full of hard codes.

I fully understand this is not ideal, but given the lack of native support for streaming large files in Feign, I had no other choice.

I hope the Feign team can consider adding native support for streaming large files in the future. This would not only address issues like ours but also make Feign more robust for handling various HTTP use cases.

Thank you for your consideration.