Skip to content

Commit

Permalink
Adds support for thread interruption in compilation and execution (#1211
Browse files Browse the repository at this point in the history
)

* Adds support for thread interruption in compilation and execution

* Adds support for CLI users to use CTRL-C
  • Loading branch information
johnedquinn authored Sep 19, 2023
1 parent 36aeaa0 commit 1c08148
Show file tree
Hide file tree
Showing 11 changed files with 838 additions and 122 deletions.
4 changes: 4 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,10 @@ Thank you to all who have contributed!
## [Unreleased]

### Added
- Adds `isInterruptible` property to `CompileOptions`. The default value is `false`. Please see the KDocs for more information.
- Adds support for thread interruption in compilation and execution. If you'd like to opt-in to this addition, please see
the `isInterruptible` addition above for more information.
- Adds support for CLI users to use CTRL-C to cancel long-running compilation/execution of queries

### Changed

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -145,6 +145,7 @@ internal sealed class AbstractPipeline(open val options: PipelineOptions) {
projectionIteration(options.projectionIterationBehavior)
undefinedVariable(options.undefinedVariableBehavior)
typingMode(options.typingMode)
isInterruptible(true)
}

private val compilerPipeline = CompilerPipeline.build {
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
/*
* Copyright Amazon.com, Inc. or its affiliates. All rights reserved.
*
* Licensed under the Apache License, Version 2.0 (the "License").
* You may not use this file except in compliance with the License.
* A copy of the License is located at:
*
* http://aws.amazon.com/apache2.0/
*
* or in the "license" file accompanying this file. This file is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific
* language governing permissions and limitations under the License.
*/

package org.partiql.cli.shell

import org.partiql.cli.pipeline.AbstractPipeline
import org.partiql.lang.eval.EvaluationSession
import org.partiql.lang.eval.PartiQLResult
import java.util.concurrent.BlockingQueue
import java.util.concurrent.TimeUnit
import java.util.concurrent.atomic.AtomicBoolean

/**
* A wrapper over [AbstractPipeline]. It constantly grabs input queries from [inputs] and places the [PartiQLResult]
* in [results]. When it is done compiling a single statement, it sets [doneCompiling] to true.
*/
internal class RunnablePipeline(
private val inputs: BlockingQueue<Input>,
private val results: BlockingQueue<PartiQLResult>,
val pipeline: AbstractPipeline,
private val doneCompiling: AtomicBoolean
) : Runnable {
/**
* When the Thread running this [Runnable] is interrupted, the underlying [AbstractPipeline] should catch the
* interruption and fail with some exception. Then, this will break out of [run].
*/
override fun run() {
while (true) {
val input = inputs.poll(3, TimeUnit.SECONDS)
if (input != null) {
val result = pipeline.compile(input.input, input.session)
results.put(result)
doneCompiling.set(true)
}
}
}

/**
* Represents a PartiQL statement ([input]) and the [EvaluationSession] to evaluate with.
*/
internal data class Input(
val input: String,
val session: EvaluationSession
)
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
/*
* Copyright Amazon.com, Inc. or its affiliates. All rights reserved.
*
* Licensed under the Apache License, Version 2.0 (the "License").
* You may not use this file except in compliance with the License.
* A copy of the License is located at:
*
* http://aws.amazon.com/apache2.0/
*
* or in the "license" file accompanying this file. This file is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific
* language governing permissions and limitations under the License.
*/

package org.partiql.cli.shell

import org.partiql.lang.eval.ExprValue
import org.partiql.lang.util.ExprValueFormatter
import java.io.PrintStream
import java.util.concurrent.BlockingQueue
import java.util.concurrent.TimeUnit
import java.util.concurrent.atomic.AtomicBoolean

/**
* A wrapper over [ExprValueFormatter]. It constantly grabs [ExprValue]s from [values] and writes them to [out].
* When it is done printing a single item, it sets [donePrinting] to true.
*/
internal class RunnableWriter(
private val out: PrintStream,
private val formatter: ExprValueFormatter,
private val values: BlockingQueue<ExprValue>,
private val donePrinting: AtomicBoolean
) : Runnable {

/**
* When the Thread running this [Runnable] is interrupted, the underlying formatter should check the interruption
* flag and fail with some exception. The formatter itself doesn't do this, but, since [ExprValue]s are lazily created,
* the creation of the [ExprValue] (by means of the thunks produced by the EvaluatingCompiler) should throw an exception
* when the thread is interrupted. Then, this will break out of [run].
*/
override fun run() {
while (true) {
val value = values.poll(3, TimeUnit.SECONDS)
if (value != null) {
out.info(BAR_1)
formatter.formatTo(value, out)
out.println()
out.info(BAR_2)
donePrinting.set(true)
}
}
}
}
Loading

1 comment on commit 1c08148

@github-actions
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

JMH Benchmark

Benchmark suite Current: 1c08148 Previous: 36aeaa0 Ratio
org.partiql.jmh.benchmarks.CompilerInterruptionBenchmark.compileCrossJoinAggFuncGroupingWithInterruptible 385.8535749750248 us/op
org.partiql.jmh.benchmarks.CompilerInterruptionBenchmark.compileCrossJoinAggFuncGroupingWithoutInterruptible 385.0247868482748 us/op
org.partiql.jmh.benchmarks.CompilerInterruptionBenchmark.compileCrossJoinAggFuncWithInterruptible 336.3341732555503 us/op
org.partiql.jmh.benchmarks.CompilerInterruptionBenchmark.compileCrossJoinAggFuncWithoutInterruptible 317.76214049290303 us/op
org.partiql.jmh.benchmarks.CompilerInterruptionBenchmark.compileCrossJoinWithInterruptible 239.11189052612275 us/op
org.partiql.jmh.benchmarks.CompilerInterruptionBenchmark.compileCrossJoinWithoutInterruptible 242.78654591342274 us/op
org.partiql.jmh.benchmarks.CompilerInterruptionBenchmark.evalCrossJoinAggWithInterruptible 7490552.546200002 us/op
org.partiql.jmh.benchmarks.CompilerInterruptionBenchmark.evalCrossJoinAggWithoutInterruptible 7613467.579600001 us/op
org.partiql.jmh.benchmarks.CompilerInterruptionBenchmark.evalCrossJoinWithInterruptible 62.66471346218746 us/op
org.partiql.jmh.benchmarks.CompilerInterruptionBenchmark.evalCrossJoinWithoutInterruptible 59.409830399312554 us/op
org.partiql.jmh.benchmarks.CompilerInterruptionBenchmark.iterCrossJoinAggWithInterruptible 8672290.6677 us/op
org.partiql.jmh.benchmarks.CompilerInterruptionBenchmark.iterCrossJoinAggWithoutInterruptible 8540773.668149998 us/op
org.partiql.jmh.benchmarks.CompilerInterruptionBenchmark.iterCrossJoinWithInterruptible 152606.64315714285 us/op
org.partiql.jmh.benchmarks.CompilerInterruptionBenchmark.iterCrossJoinWithoutInterruptible 153569.73290714287 us/op
org.partiql.jmh.benchmarks.MultipleLikeBenchmark.testPartiQLCompiler15 138.79642778507522 us/op 181.18106429412893 us/op 0.77
org.partiql.jmh.benchmarks.MultipleLikeBenchmark.testPartiQLCompiler30 261.0646059014297 us/op 265.86675633448124 us/op 0.98
org.partiql.jmh.benchmarks.MultipleLikeBenchmark.testPartiQLEvaluator15 649932.346375 us/op 577989.45175 us/op 1.12
org.partiql.jmh.benchmarks.MultipleLikeBenchmark.testPartiQLEvaluator30 1308399.0264 us/op 1319299.06445 us/op 0.99
org.partiql.jmh.benchmarks.MultipleLikeBenchmark.testPartiQLEvaluator30WithData10 12828998.287749996 us/op 13120971.443200003 us/op 0.98
org.partiql.jmh.benchmarks.MultipleLikeBenchmark.testPartiQLParser15 213.128718777517 us/op 223.9413677464257 us/op 0.95
org.partiql.jmh.benchmarks.MultipleLikeBenchmark.testPartiQLParser30 426.0383206185035 us/op 426.9564562299852 us/op 1.00
org.partiql.jmh.benchmarks.ParserBenchmark.parseFailNameCaseWhenThen 53.06810354547751 us/op 55.20301048729499 us/op 0.96
org.partiql.jmh.benchmarks.ParserBenchmark.parseFailNameComplexQuery 73.81562257843501 us/op 75.01881928206943 us/op 0.98
org.partiql.jmh.benchmarks.ParserBenchmark.parseFailNameComplexQuery01 371.62660599353524 us/op 414.530518015821 us/op 0.90
org.partiql.jmh.benchmarks.ParserBenchmark.parseFailNameComplexQuery02 649.1550583441019 us/op 676.2049115465617 us/op 0.96
org.partiql.jmh.benchmarks.ParserBenchmark.parseFailNameExceptUnionIntersectSixty 253.52142286237626 us/op 257.6093298848457 us/op 0.98
org.partiql.jmh.benchmarks.ParserBenchmark.parseFailNameExec20Expressions 83.53422049321019 us/op 86.9412585303792 us/op 0.96
org.partiql.jmh.benchmarks.ParserBenchmark.parseFailNameFromLet 55.06186344757632 us/op 57.104463223017376 us/op 0.96
org.partiql.jmh.benchmarks.ParserBenchmark.parseFailNameGraphPattern 53.83358116710449 us/op 57.76604654404233 us/op 0.93
org.partiql.jmh.benchmarks.ParserBenchmark.parseFailNameGraphPreFilters 87.3576287154029 us/op 91.01982532931856 us/op 0.96
org.partiql.jmh.benchmarks.ParserBenchmark.parseFailNameGroupLimit 59.81661748789422 us/op 61.195328483499225 us/op 0.98
org.partiql.jmh.benchmarks.ParserBenchmark.parseFailNameLongFromSourceOrderBy 85.47104678556799 us/op 82.47104800537008 us/op 1.04
org.partiql.jmh.benchmarks.ParserBenchmark.parseFailNameManyJoins 81.08687685900097 us/op 83.79708255458681 us/op 0.97
org.partiql.jmh.benchmarks.ParserBenchmark.parseFailNameNestedAggregates 142.7514566836876 us/op 140.81129981233502 us/op 1.01
org.partiql.jmh.benchmarks.ParserBenchmark.parseFailNameNestedParen 25.940067371253672 us/op 26.560068660918233 us/op 0.98
org.partiql.jmh.benchmarks.ParserBenchmark.parseFailNamePivot 87.26420585049416 us/op 91.53793150227231 us/op 0.95
org.partiql.jmh.benchmarks.ParserBenchmark.parseFailNameQuery15OrsAndLikes 280.13810929384005 us/op 270.9971827682097 us/op 1.03
org.partiql.jmh.benchmarks.ParserBenchmark.parseFailNameQuery30Plus 142.465359702656 us/op 153.1995155708268 us/op 0.93
org.partiql.jmh.benchmarks.ParserBenchmark.parseFailNameQueryFunc 66.78821741775917 us/op 68.41277543067952 us/op 0.98
org.partiql.jmh.benchmarks.ParserBenchmark.parseFailNameQueryFuncInProjection 148.57273257176985 us/op 153.99598618380097 us/op 0.96
org.partiql.jmh.benchmarks.ParserBenchmark.parseFailNameQueryList 101.4367508122031 us/op 102.78023163351133 us/op 0.99
org.partiql.jmh.benchmarks.ParserBenchmark.parseFailNameQueryNestedSelect 900.9716389465286 us/op 957.0940803867179 us/op 0.94
org.partiql.jmh.benchmarks.ParserBenchmark.parseFailNameQuerySimple 23.078439816799495 us/op 23.310846645219364 us/op 0.99
org.partiql.jmh.benchmarks.ParserBenchmark.parseFailNameSeveralJoins 30.626961741320024 us/op 32.04910493514403 us/op 0.96
org.partiql.jmh.benchmarks.ParserBenchmark.parseFailNameSeveralProjections 95.6484017742529 us/op 100.06551516671036 us/op 0.96
org.partiql.jmh.benchmarks.ParserBenchmark.parseFailNameSeveralSelect 254.8188926334796 us/op 261.9873564421158 us/op 0.97
org.partiql.jmh.benchmarks.ParserBenchmark.parseFailNameSimpleInsert 39.04119527831246 us/op 42.82326469908479 us/op 0.91
org.partiql.jmh.benchmarks.ParserBenchmark.parseFailNameSomeJoins 30.23783280995039 us/op 31.10649190583148 us/op 0.97
org.partiql.jmh.benchmarks.ParserBenchmark.parseFailNameSomeProjections 41.43465531484775 us/op 40.9725298726896 us/op 1.01
org.partiql.jmh.benchmarks.ParserBenchmark.parseFailNameSomeSelect 66.6475227025011 us/op 68.44734034579042 us/op 0.97
org.partiql.jmh.benchmarks.ParserBenchmark.parseFailNameTimeZone 33.701586677229876 us/op 34.870218153244686 us/op 0.97
org.partiql.jmh.benchmarks.ParserBenchmark.parseFailNameVeryLongQuery 425.5501022114762 us/op 430.4435595830628 us/op 0.99
org.partiql.jmh.benchmarks.ParserBenchmark.parseFailNameVeryLongQuery01 1254.0653777919294 us/op 1308.3715027955527 us/op 0.96
org.partiql.jmh.benchmarks.ParserBenchmark.parseNameCaseWhenThen 34.620676474023135 us/op 35.38357220114894 us/op 0.98
org.partiql.jmh.benchmarks.ParserBenchmark.parseNameComplexQuery 296.2286651409627 us/op 307.06894949770384 us/op 0.96
org.partiql.jmh.benchmarks.ParserBenchmark.parseNameComplexQuery01 136.63806825853206 us/op 143.638882864122 us/op 0.95
org.partiql.jmh.benchmarks.ParserBenchmark.parseNameExceptUnionIntersectSixty 285.93933484482034 us/op 301.2048913046468 us/op 0.95
org.partiql.jmh.benchmarks.ParserBenchmark.parseNameExec20Expressions 84.9823886897242 us/op 89.87822446052462 us/op 0.95
org.partiql.jmh.benchmarks.ParserBenchmark.parseNameFromLet 47.18553500646068 us/op 48.0940238920637 us/op 0.98
org.partiql.jmh.benchmarks.ParserBenchmark.parseNameGraphPattern 53.72661535394243 us/op 56.03193041413518 us/op 0.96
org.partiql.jmh.benchmarks.ParserBenchmark.parseNameGraphPreFilters 88.31602963082018 us/op 92.26027608558712 us/op 0.96
org.partiql.jmh.benchmarks.ParserBenchmark.parseNameGroupLimit 45.162111048322274 us/op 44.87633103435974 us/op 1.01
org.partiql.jmh.benchmarks.ParserBenchmark.parseNameLongFromSourceOrderBy 158.51136631747437 us/op 162.59090535902445 us/op 0.97
org.partiql.jmh.benchmarks.ParserBenchmark.parseNameManyJoins 56.34421514951434 us/op 57.80230177235104 us/op 0.97
org.partiql.jmh.benchmarks.ParserBenchmark.parseNameNestedAggregates 115.62143417116565 us/op 121.65524810084564 us/op 0.95
org.partiql.jmh.benchmarks.ParserBenchmark.parseNameNestedParen 100.73957765907114 us/op 103.04557657902824 us/op 0.98
org.partiql.jmh.benchmarks.ParserBenchmark.parseNamePivot 83.82937133602431 us/op 84.61696497124572 us/op 0.99
org.partiql.jmh.benchmarks.ParserBenchmark.parseNameQuery15OrsAndLikes 223.95032592103007 us/op 236.49464442527506 us/op 0.95
org.partiql.jmh.benchmarks.ParserBenchmark.parseNameQuery30Plus 78.9723665476191 us/op 81.47657379205913 us/op 0.97
org.partiql.jmh.benchmarks.ParserBenchmark.parseNameQueryFunc 170.30023290981063 us/op 171.89528028110058 us/op 0.99
org.partiql.jmh.benchmarks.ParserBenchmark.parseNameQueryFuncInProjection 114.69343478636577 us/op 120.92504278435482 us/op 0.95
org.partiql.jmh.benchmarks.ParserBenchmark.parseNameQueryList 99.99833056666472 us/op 105.99740794790895 us/op 0.94
org.partiql.jmh.benchmarks.ParserBenchmark.parseNameQueryNestedSelect 171.6880904640219 us/op 170.61139809884605 us/op 1.01
org.partiql.jmh.benchmarks.ParserBenchmark.parseNameQuerySimple 16.70466712812266 us/op 16.864769862348034 us/op 0.99
org.partiql.jmh.benchmarks.ParserBenchmark.parseNameSeveralJoins 95.90867660017943 us/op 95.37563359962596 us/op 1.01
org.partiql.jmh.benchmarks.ParserBenchmark.parseNameSeveralProjections 70.92193735892593 us/op 70.55213978131195 us/op 1.01
org.partiql.jmh.benchmarks.ParserBenchmark.parseNameSeveralSelect 140.7547076572051 us/op 142.6400755544531 us/op 0.99
org.partiql.jmh.benchmarks.ParserBenchmark.parseNameSimpleInsert 29.149893364432103 us/op 28.68591161964835 us/op 1.02
org.partiql.jmh.benchmarks.ParserBenchmark.parseNameSomeJoins 26.921978512488995 us/op 29.546454005651093 us/op 0.91
org.partiql.jmh.benchmarks.ParserBenchmark.parseNameSomeProjections 23.998781086418155 us/op 24.406872184659598 us/op 0.98
org.partiql.jmh.benchmarks.ParserBenchmark.parseNameSomeSelect 44.156139256845115 us/op 45.424064168305165 us/op 0.97
org.partiql.jmh.benchmarks.ParserBenchmark.parseNameTimeZone 11.762177931999044 us/op 12.225044447715304 us/op 0.96
org.partiql.jmh.benchmarks.ParserBenchmark.parseNameVeryLongQuery 549.3721635099971 us/op 561.4633030979722 us/op 0.98
org.partiql.jmh.benchmarks.ParserBenchmark.parseNameVeryLongQuery01 1552.6088311939543 us/op 1530.6975949236391 us/op 1.01
org.partiql.jmh.benchmarks.PartiQLBenchmark.testPartiQLCompiler 11.96680076996124 us/op 12.78703403925275 us/op 0.94
org.partiql.jmh.benchmarks.PartiQLBenchmark.testPartiQLEvaluator 3.1513331163405107 us/op 3.223388168934926 us/op 0.98
org.partiql.jmh.benchmarks.PartiQLBenchmark.testPartiQLParser 15.425383865793256 us/op 16.07503737456039 us/op 0.96

This comment was automatically generated by workflow using github-action-benchmark.

Please sign in to comment.