Server configuration can be provided as part of the server startup parameters.
bin/pinot-admin.sh StartServer -configFileName /path/to/server.conf
server.conf can have the following properties
| Property | Default | Description |
|---|---|---|
| pinot.server.netty.port | 8098 | Port to query Pinot Server |
| pinot.server.netty.host | Pinot server hostname | |
| pinot.server.adminapi.port | 8097 | Port for Pinot Server Admin UI |
| pinot.server.instance.id | By default the server instance id used by Helix is Server_hostname_port where the hostname and port are configured through host and port config values above. This config overwrites the default setting. User can put server id independent of the server's hostname and port. | |
| pinot.server.instance.dataDir | java.io.tmpdir + /PinotServer/index |
Directory to hold all the data |
| pinot.server.instance.consumerDir | ||
| pinot.server.instance.segmentTarDir | java.io.tmpdir + /PinotServer/segmentTar |
Directory to hold temporary segments downloaded from Controller or Deep Store |
| pinot.server.instance.readMode | mmap |
|
| pinot.server.instance.reload.consumingSegment | true | Specifies if the reload segment API should reload the consuming segments. This is useful when the corresponding schema is updated and we want the changes to be reflected in the consuming segment. |
| pinot.server.instance.data.manager.class |
|
|
| pinot.query.scheduler.name | fcfs |
Currently only FCFS (first-come-first-serve) is supported |
| pinot.query.scheduler.query_runner_threads | CPU cores | Main thread to execute the queries (one thread per query) |
| pinot.query.scheduler.query_worker_threads | 2 * CPU cores | Worker thread to process the segments (multiple threads per query) |
| pinot.query.scheduler.query.log.maxRatePerSecond | unlimited | Maximum queries to be logged per second. Queries with exceptions and costly queries are always logged. |
| pinot.server.query.executor.class |
|
|
| pinot.server.query.executor.pruner.class |
|
|
| pinot.server.query.executor.timeout | 15000 |
Timeout for Server to process Query in Milliseconds |
| pinot.server.query.executor.max.execution.threads | -1 (unlimited) |
Maximum number of execution threads allowed for a query. Limiting this can prevent a single expensive query from occupying all the execution threads. |
| pinot.server.query.executor.max.init.group.holder.capacity | 10000 |
Initial capacity of the group key holder. Increasing this value can reduce the resizing of the group key holder, but increase the heap usage for small group-by queries. |
| pinot.server.query.executor.num.groups.limit | 100000 |
Maximum number of groups kept from each segment during query execution. Once this limit is reached, no more groups will be taken (will still aggregate on existing groups). |
| pinot.server.query.executor.min.segment.group.trim.size | -1 (do not trim) |
Minimum number of groups kept at segment level during query execution. If there are enough groups found in a segment, pinot will trim the groups to max(5 * LIMIT, minSegmentgroupTrimSize) groups based on the order-by clause. Increasing this value can increase the accuracy of the results, but also increase the heap usage of the group-by queries. |
| pinot.server.query.executor.min.server.group.trim.size | 5000 |
Minimum number of groups kept at server level during query execution. If there are enough groups found in a server, pinot will trim the groups to max(5 * LIMIT, minServergroupTrimSize) groups based on the order-by clause. Increasing this value can increase the accuracy of the results, but also increase the heap usage and data transfer cost of the group-by queries. |
| pinot.server.query.executor.groupby.trim.threshold | 1000000 |
Threshold for number of groups to trigger the server group trimming. Increasing this value can reduce the times of trimming, but also increase the heap usage of the group-by queries. |
| pinot.server.requestHandlerFactory.class |
|
|
| pinot.server.instance.segment.format.version | ||
| pinot.server.instance.enable.split.commit | ||
| pinot.server.instance.enable.commitend.metadata | ||
| pinot.server.instance.max.parallel.refresh.threads | 1 | Number of simultaneous segments that can be refreshed on one server. |
| pinot.server.instance.realtime.max.parallel.segment.builds | 0 | Specifies how many parallel real-time segments can be built. Value of <= 0 indicates unlimited. |
| pinot.server.instance.realtime.alloc.offheap | true | Boolean value to control whether memory for real-time consuming segments should be allocated off-heap. |
| pinot.server.instance.realtime.alloc.offheap.direct | false | If 'real-time.alloc.offheap' is set to true, this boolean value controls whether the corresponding allocation should be direct or not (false indicate mmap allocation) |
| pinot.server.startup.minResourcePercent | 100 | The percentage of tables that need to be in an ONLINE state before the server is marked as STARTED to server queries |
| pinot.server.startup.timeoutMs | 10 minutes | The total amount of time a server will wait for all status checks before server is marked as STARTED to server queries |
| pinot.server.starter.realtimeConsumptionCatchupWaitMs | 0 | On it's own, this is a static amount of time servers will wait for consuming segments before server is marked as |
| pinot.server.starter.enableRealtimeOffsetBasedConsumptionStatusChecker | false | When true, the server will collect the current, latest offsets for all consuming segments and will mark those segments as GOOD once they have caught up to those offsets |
| pinot.server.starter.enableRealtimeFreshnessBasedConsumptionStatusChecker | false | When
|
| pinot.server.starter.realtimeMinFreshnessMs | 10 seconds | When This only applies when |
| pinot.server.starter.realtimeFreshnessIdleTimeoutMs | 0 | When pinot.server.starter.enableRealtimeFreshnessBasedConsumptionStatusChecker=true, this will mark a segment as GOOD after this timeout if it has not consumed any events. |
| pinot.server.startup.exitOnServiceStatusCheckFailure | false | When true, the server will shutdown after pinot.server.shutdown.timeoutMs rather than be marked as STARTED if the status has not turned GOOD |
| pinot.server.startup.enableServiceStatusCheck | true |
|
| pinot.server.startup.serviceStatusCheckIntervalMs | 10 seconds | Interval at which server will perform service stats checks |
| pinot.server.shutdown.timeoutMs | 10 minutes | |
| pinot.server.shutdown.enableQueryCheck | true | |
| pinot.server.shutdown.noQueryThresholdMs | 15 seconds | |
| pinot.server.shutdown.enableResourceCheck | false | |
| pinot.server.shutdown.resourceCheckIntervalMs | 10 seconds | |
| pinot.server.admin.access.control.factory.class |
|
|
| pinot.server.adminapi.access.protocols | http | Ingress protocols to access server admin api (http or https or http,https) |
| pinot.server.adminapi.access.protocols.http.port | Port to access server admin api via http | |
| pinot.server.adminapi.broker.protocols.https.port | Port to access server admin api via https | |
| pinot.server.tls.keystore.path | Path to server TLS keystore | |
| pinot.server.tls.keystore.password | keystore password | |
| pinot.server.tls.truststore.path | Path to server TLS truststore | |
| pinot.server.tls.truststore.password | truststore password | |
| inot.server.tls.client.auth | false | toggle for requiring TLS client auth |
| pinot.server.netty.enabled | true | toggle for enabling unsecured netty connections to server |
| pinot.server.netty.port | Port for accessing pinot server via unsecured netty | |
| pinot.server.nettytls.enabled | ||
| pinot.server.nettytls.port | Port for accessing pinot server via TLS-secured netty | |
| pinot.server.http.server.thread.pool.corePoolSize | 2 * cores | Config for the thread-pool used by pinot-server's http-server. |
| pinot.server.http.server.thread.pool.maxPoolSize | 2 * cores | Config for the thread-pool used by pinot-server's http-server. |
| pinot.server.segment.fetcher.http.client.maxConnTotal | Config for the http-client used by HttpSegmentFetcher for downloading segments | |
| pinot.server.segment.fetcher.http.client.maxConnPerRoute | Config for the http-client used by HttpSegmentFetcher for downloading segments | |
| pinot.server.instance.max.segment.preload.threads | 0 | Number of threads that should be created to preload the segments in an upsert table. |
| pinot.server.query.regex.class | JAVA_UTIL | Determine the regex class used for query execution. Valid options are JAVA_UTIL and RE2J |
| pinot.server.instance.consumer.client.id.suffix | Configures a suffix to differentiate consumer instances. When multiple replicas of a real-time consuming segment have the same client ID (formatted as TableName-TopicName-Partition), this suffix can be used add a unique server level identity. |
|
| pinot.server.segment.uploader.protocol | http | Configure the http protocol when server upload segments to controller. This is required |
| pinot.server.segment.uploader.https.enabled | false | Enable https for server upload protocol, note that you need to configure pinot.server.segment.uploader.protocol=https |
| pinot.server.segment.uploader.https.ssl.xxxx | All the ssl related configures goes here | |
| pinot.server.segment.uploader.upload.request.timeout.ms | 300_000 | default is 300 seconds/5mins |
| pinot.server.segment.uploader.auth.xxx | Auth related configs go here | |
| pinot.server.segment.uploader.http.client.maxConnTotal | uploader http client config | |
| pinot.server.segment.uploader.http.client.maxConnPerRoute | uploader http client config | |
| pinot.server.segment.uploader.http.client.disableDefaultUserAgent | uploader http client config |