Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Set fetch size for JdbcHelper #63

Open
fabianMendez opened this issue Oct 20, 2020 · 4 comments
Open

Set fetch size for JdbcHelper #63

fabianMendez opened this issue Oct 20, 2020 · 4 comments
Assignees
Labels
enhancement New feature or request

Comments

@fabianMendez
Copy link

Hi there,

I was testing this library to generate a quite large report with over ~150k rows from an SQL database, one of the issues I found was that it was taking so much time (~3 minutes) and it was due to the rather low value of rows fetched per network call, ie: if no value is specified using the setFetchSize method of the statement, it fetches 10 rows at a time by default, which means over 15k network calls to the database.

Once I overwrote the query method from the JdbcHelper class and specified 1000 rows as the fetch size, the same report took only ~25s.

So I am proposing to create a new constructor for the JdbcHelper class which allows to specify the fetchSize to use. The implementation is pretty simple so I can create the PR, but I am not sure if this can be tested because the the in-memory database probably won't be affected by a network optimization.

@fabianMendez fabianMendez added the enhancement New feature or request label Oct 20, 2020
@SoltauFintel
Copy link
Member

It is already possible to specify the fetch size for the connection as a connection property.

Oracle=defaultRowPrefetch, PostgreSQL=defaultRowFetchSize, MySQL/MariaDB=defaultFetchSize, DB2=fetchSize

@fabianMendez & @leonate - What is your opinion?

@fabianMendez
Copy link
Author

That would be an option, however, I am not sure if it is possible to specify such properties without using the JDBC URL, because that would affect the whole application using the connection.

According to this, it is possible but is driver specific, so one would have to check if the connection is an instance of a class to use the specific method for that class.

In short, I think it would be a lot of less hassle if it the JdbcHelper class supports it as an optional parameter of the constructor.

@SoltauFintel
Copy link
Member

@leonate - what's your opinion?

@leonate
Copy link
Member

leonate commented Feb 21, 2021

An option to set the fetch size makes sense to me. Need to take a look what changes would be needed to support it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

3 participants