diff options
| author | Android Build Coastguard Worker <android-build-coastguard-worker@google.com> | 2023-08-10 20:10:44 +0000 |
|---|---|---|
| committer | Android Build Coastguard Worker <android-build-coastguard-worker@google.com> | 2023-08-10 20:10:44 +0000 |
| commit | 178a92b53f8b71e5645f5321f5697f4210799ad5 (patch) | |
| tree | c666b5a2e756f00ff3a7885b942e2ab4b35362eb | |
| parent | 473ce1031e6aaa72b5999411e26721df136853b8 (diff) | |
| parent | e2ece58ebfecc731282410888c0d3e983616285f (diff) | |
| download | kotlinx.coroutines-178a92b53f8b71e5645f5321f5697f4210799ad5.tar.gz | |
Snap for 10641560 from e2ece58ebfecc731282410888c0d3e983616285f to simpleperf-release
Change-Id: I976efed37fd0b5a3cc6a7a193ea8e95d83bac2cf
480 files changed, 16022 insertions, 9003 deletions
diff --git a/.github/ISSUE_TEMPLATE/bug_report.md b/.github/ISSUE_TEMPLATE/bug_report.md new file mode 100644 index 00000000..0fa80d1f --- /dev/null +++ b/.github/ISSUE_TEMPLATE/bug_report.md @@ -0,0 +1,29 @@ +--- +name: Bug report +about: Our code behaves incorrectly? +title: '' +labels: bug +assignees: '' + +--- + +<!-- +**Double-check** + +* Is this *really* a bug? + - If the behavior is documented, but you disagree with it, please do file an issue, but as a "Design consideration," not a "Bug report." + - If you don't understand why something behaves the way it does, consider asking on [StackOverflow](https://stackoverflow.com/) or the [Kotlin Slack](https://surveys.jetbrains.com/s3/kotlin-slack-sign-up). The community is active and will likely clarify everything better than we could! +* Is the problem not in some third-party library, not in [Kotlin](kotl.in/issue), or your own code—is it in the `kotlinx.coroutines` library itself? + - Example: you write for Android, and your code works properly on most devices, but for a couple of them, it fails. Then please direct this to Google and/or the manufacturer of your device. +* Maybe you're using some ancient version, and the problem doesn't happen with the latest releases of the compiler and the library? +--> + +**Describe the bug** + +What happened? What should have happened instead? + +**Provide a Reproducer** + +* If possible, please provide a small self-contained project (or even just a single file) where the issue reproduces. +* If you can't pinpoint the issue, please provide at least *some* project where this reproduces, for example, your production one. If you are not ready to show the project publicly, we are open to discussing the details privately. +* If you really can't provide any code, please do still open an issue. This may prompt other people to chime in with their reproducers. diff --git a/.github/ISSUE_TEMPLATE/config.yml b/.github/ISSUE_TEMPLATE/config.yml new file mode 100644 index 00000000..145cd391 --- /dev/null +++ b/.github/ISSUE_TEMPLATE/config.yml @@ -0,0 +1,5 @@ +blank_issues_enabled: true +contact_links: + - name: Kotlinlang Slack + url: https://surveys.jetbrains.com/s3/kotlin-slack-sign-up + about: Please ask and answer usage-related questions here. diff --git a/.github/ISSUE_TEMPLATE/design_considerations.md b/.github/ISSUE_TEMPLATE/design_considerations.md new file mode 100644 index 00000000..cca209d8 --- /dev/null +++ b/.github/ISSUE_TEMPLATE/design_considerations.md @@ -0,0 +1,45 @@ +--- +name: Design considerations +about: We didn't think things through? +title: '' +labels: design +assignees: '' + +--- + +<!-- +This is a place for issue reports that are not exactly bugs (wrong unintentional behavior) but for our library behaving suboptimally (though this could have been intentional). + +**Double-check** + +* If the behavior is strange, surprising, and undocumented, it could be a good idea to file a "Bug report" instead. +* Is this still relevant with the latest version of the library? We could have changed this already. +* Maybe there are good reasons for the existing behavior. Please try searching for existing discussions of the problem. +* Are you using the right abstraction? Consider asking on [StackOverflow](https://stackoverflow.com/) or the [Kotlin Slack](https://surveys.jetbrains.com/s3/kotlin-slack-sign-up). Maybe your need is better solved by some other abstraction entirely. + +--> + +**What do we have now?** + +Preferably with specific code examples. + +**What should be instead?** + +Preferably with specific code examples. + +**Why?** + +The upsides of your proposal. +* Who would benefit from this and how? + - Would it be possible to cover new use cases? + - Would some code become clearer? + - Would the library become conceptually simpler? + - etc. + +**Why not?** + +The downsides of your proposal that you already see. +* Is this a breaking change? +* Are there use cases that are better solved by what we have now? +* Does some code become less clear after this change? +* etc. diff --git a/.github/ISSUE_TEMPLATE/feature_request.md b/.github/ISSUE_TEMPLATE/feature_request.md new file mode 100644 index 00000000..4fb7a0b7 --- /dev/null +++ b/.github/ISSUE_TEMPLATE/feature_request.md @@ -0,0 +1,33 @@ +--- +name: Feature request +about: We're missing something? +title: '' +labels: enhancement +assignees: '' + +--- + +<!-- +**Double-check** + +* Maybe this feature is already here? + - Did you check the latest version of the library? + - Maybe it's in a form you didn't expect? Consider asking on [StackOverflow](https://stackoverflow.com/) or the [Kotlin Slack](https://surveys.jetbrains.com/s3/kotlin-slack-sign-up). The community will likely come up with some code that solves your need, and faster than it would take us to answer the issue! +* Do you actually *need* this feature? Maybe restructuring your code would neatly eliminate the problem the feature would be solving. +* Is the coroutines library the best place for this feature? Maybe it would be better suited for some third-party library? +--> + +**Use case** + +Explain what *specifically* you are trying to do and why. +- Example: "I have a `SharedFlow<Double>` that represents readings from an external device. The readings arrive in a set interval of 100 milliseconds. However, I also need to be able to calibrate the state of the external device by setting the readings from inside the program. When I set the state, the `SharedFlow<Double>` must immediately emit the value that was set and ignore any values coming from the device in the following 10 milliseconds since they are considered outdated, as the device is only guaranteed to recalibrate to the updated value after that period." +- Non-example: "I have a `SharedFlow<T>` that has several sources of its values, and these sources need to have priorities attached to them so that one source always takes precedence over the other in a close race." +- Non-example: "RxJava has feature X, so the coroutines library should also." + +**The Shape of the API** + +What could the desired API look like? What would some sample code using the new feature look like? If you don't have a clear idea, pseudocode or just explaining the API shape is also perfectly fine. + +**Prior Art** + +(Optional) Maybe you have seen something like the feature you need, but in other libraries, or there is something very similar but not quite sufficient in `kotlinx.coroutines`? Maybe there's already a way to do it, but it's too cumbersome and unclear? @@ -12,4 +12,5 @@ build out target local.properties +benchmarks.jar /kotlin-js-store diff --git a/.idea/codeStyles/Project.xml b/.idea/codeStyles/Project.xml index 62fd5c7d..461a31ed 100644 --- a/.idea/codeStyles/Project.xml +++ b/.idea/codeStyles/Project.xml @@ -12,4 +12,4 @@ </indentOptions> </codeStyleSettings> </code_scheme> -</component>
\ No newline at end of file +</component> diff --git a/.idea/dictionaries/shared.xml b/.idea/dictionaries/shared.xml index 3da8e229..45cbedcf 100644 --- a/.idea/dictionaries/shared.xml +++ b/.idea/dictionaries/shared.xml @@ -1,8 +1,13 @@ <component name="ProjectDictionaryState"> <dictionary name="shared"> <words> + <w>Alistarh</w> + <w>Elizarov</w> + <w>Koval</w> <w>kotlinx</w> <w>lincheck</w> + <w>linearizability</w> + <w>linearizable</w> <w>redirector</w> </words> </dictionary> @@ -47,7 +47,6 @@ java_library_host { "kotlinx-coroutines-core/concurrent/src/**/*.kt", ], exclude_srcs: [ - "kotlinx-coroutines-core/jvm/src/debug/**/*.kt", "kotlinx-coroutines-core/jvm/src/test_/TestCoroutineContext.kt", ], java_resource_dirs: ["kotlinx-coroutines-core/jvm/resources"], @@ -1,1463 +1,88 @@ # Change log for kotlinx.coroutines -## Version 1.6.4 - -* Added `TestScope.backgroundScope` for launching coroutines that perform work in the background and need to be cancelled at the end of the test (#3287). -* Fixed the POM of `kotlinx-coroutines-debug` having an incorrect reference to `kotlinx-coroutines-bom`, which cause the builds of Maven projects using the debug module to break (#3334). -* Fixed the `Publisher.await` functions in `kotlinx-coroutines-reactive` not ensuring that the `Subscriber` methods are invoked serially (#3360). Thank you, @EgorKulbachka! -* Fixed a memory leak in `withTimeout` on K/N with the new memory model (#3351). -* Added the guarantee that all `Throwable` implementations in the core library are serializable (#3328). -* Moved the documentation to <https://kotlinlang.org/api/kotlinx.coroutines/> (#3342). -* Various documentation improvements. - -## Version 1.6.3 - -* Updated atomicfu version to 0.17.3 (#3321), fixing the projects using this library with JS IR failing to build (#3305). - -## Version 1.6.2 - -* Fixed a bug with `ThreadLocalElement` not being correctly updated when the most outer `suspend` function was called directly without `kotlinx.coroutines` (#2930). -* Fixed multiple data races: one that might have been affecting `runBlocking` event loop, and a benign data race in `Mutex` (#3250, #3251). -* Obsolete `TestCoroutineContext` is removed, which fixes the `kotlinx-coroutines-test` JPMS package being split between `kotlinx-coroutines-core` and `kotlinx-coroutines-test` (#3218). -* Updated the ProGuard rules to further shrink the size of the resulting DEX file with coroutines (#3111, #3263). Thanks, @agrieve! -* Atomicfu is updated to `0.17.2`, which includes a more efficient and robust JS IR transformer (#3255). -* Kotlin is updated to `1.6.21`, Gradle version is updated to `7.4.2` (#3281). Thanks, @wojtek-kalicinski! -* Various documentation improvements. - -## Version 1.6.1 - -* Rollback of time-related functions dispatching on `Dispatchers.Main`. - This behavior was introduced in 1.6.0 and then found inconvenient and erroneous (#3106, #3113). -* Reworked the newly-introduced `CopyableThreadContextElement` to solve issues uncovered after the initial release (#3227). -* Fixed a bug with `ThreadLocalElement` not being properly updated in racy scenarios (#2930). -* Reverted eager loading of default `CoroutineExceptionHandler` that triggered ANR on some devices (#3180). -* New API to convert a `CoroutineDispatcher` to a Rx scheduler (#968, #548). Thanks @recheej! -* Fixed a memory leak with the very last element emitted from `flow` builder being retained in memory (#3197). -* Fixed a bug with `limitedParallelism` on K/N with new memory model throwing `ClassCastException` (#3223). -* `CoroutineContext` is added to the exception printed to the default `CoroutineExceptionHandler` to improve debuggability (#3153). -* Static memory consumption of `Dispatchers.Default` was significantly reduced (#3137). -* Updated slf4j version in `kotlinx-coroutines-slf4j` from 1.7.25 to 1.7.32. - -## Version 1.6.0 - -Note that this is a full changelog relative to the 1.5.2 version. Changelog relative to 1.6.0-RC3 can be found at the end. - -### kotlinx-coroutines-test rework - -* `kotlinx-coroutines-test` became a multiplatform library usable from K/JVM, K/JS, and K/N. -* Its API was completely reworked to address long-standing issues with consistency, structured concurrency and correctness (#1203, #1609, #2379, #1749, #1204, #1390, #1222, #1395, #1881, #1910, #1772, #1626, #1742, #2082, #2102, #2405, #2462 - ). -* The old API is deprecated for removal, but the new API is based on the similar concepts ([README](kotlinx-coroutines-test/README.md)), and the migration path is designed to be graceful: [migration guide](kotlinx-coroutines-test/MIGRATION.md). - -### Dispatchers - -* Introduced `CoroutineDispatcher.limitedParallelism` that allows obtaining a view of the original dispatcher with limited parallelism (#2919). -* `Dispatchers.IO.limitedParallelism` usages ignore the bound on the parallelism level of `Dispatchers.IO` itself to avoid starvation (#2943). -* Introduced new `Dispatchers.shutdown` method for containerized environments (#2558). -* `newSingleThreadContext` and `newFixedThreadPoolContext` are promoted to delicate API (#2919). - -### Breaking changes - -* When racing with cancellation, the `future` builder no longer reports unhandled exceptions into the global `CoroutineExceptionHandler`. Thanks @vadimsemenov! (#2774, #2791). -* `Mutex.onLock` is deprecated for removal (#2794). -* `Dispatchers.Main` is now used as the default source of time for `delay` and `withTimeout` when present(#2972). - * To opt-out from this behaviour, `kotlinx.coroutines.main.delay` system property can be set to `false`. -* Java target of coroutines build is now 8 instead of 6 (#1589). -* **Source-breaking change**: extension `collect` no longer resolves when used with a non-in-place argument of a functional type. This is a candidate for a fix, uncovered after 1.6.0, see #3107 for the additional details. +## Version 1.7.2 ### Bug fixes and improvements -* Kotlin is updated to 1.6.0. -* Kotlin/Native [new memory model](https://blog.jetbrains.com/kotlin/2021/08/try-the-new-kotlin-native-memory-manager-development-preview/) is now supported in regular builds of coroutines conditionally depending on whether `kotlin.native.binary.memoryModel` is enabled (#2914). -* Introduced `CopyableThreadContextElement` for mutable context elements shared among multiple coroutines. Thanks @yorickhenning! (#2893). -* `transformWhile`, `awaitClose`, `ProducerScope`, `merge`, `runningFold`, `runingReduce`, and `scan` are promoted to stable API (#2971). -* `SharedFlow.subscriptionCount` no longer conflates incoming updates and gives all subscribers a chance to observe a short-lived subscription (#2488, #2863, #2871). -* `Flow` exception transparency mechanism is improved to be more exception-friendly (#3017, #2860). -* Cancellation from `flat*` operators that leverage multiple coroutines is no longer propagated upstream (#2964). -* `SharedFlow.collect` now returns `Nothing` (#2789, #2502). -* `DisposableHandle` is now `fun interface`, and corresponding inline extension is removed (#2790). -* `FlowCollector` is now `fun interface`, and corresponding inline extension is removed (#3047). -* Deprecation level of all previously deprecated signatures is raised (#3024). -* The version file is shipped with each JAR as a resource (#2941). -* Unhandled exceptions on K/N are passed to the standard library function `processUnhandledException` (#2981). -* A direct executor is used for `Task` callbacks in `kotlinx-coroutines-play-services` (#2990). -* Metadata of coroutines artifacts leverages Gradle platform to have all versions of dependencies aligned (#2865). -* Default `CoroutineExceptionHandler` is loaded eagerly and does not invoke `ServiceLoader` on its exception-handling path (#2552). -* Fixed the R8 rules for `ServiceLoader` optimization (#2880). -* Fixed BlockHound integration false-positives (#2894, #2866, #2937). -* Fixed the exception handler being invoked several times on Android, thanks to @1zaman (#3056). -* `SendChannel.trySendBlocking` is now available on Kotlin/Native (#3064). -* The exception recovery mechanism now uses `ClassValue` when available (#2997). -* JNA is updated to 5.9.0 to support Apple M1 (#3001). -* Obsolete method on internal `Delay` interface is deprecated (#2979). -* Support of deprecated `CommonPool` is removed. -* `@ExperimentalTime` is no longer needed for methods that use `Duration` (#3041). -* JDK 1.6 is no longer required for building the project (#3043). -* New version of Dokka is used, fixing the memory leak when building the coroutines and providing brand new reference visuals (https://kotlinlang.org/api/kotlinx.coroutines/) (#3051, #3054). - -### Changelog relative to version 1.6.0-RC3 - -* Restored MPP binary compatibility on K/JS and K/N (#3104). -* Fixed Dispatchers.Main not being fully initialized on Android and Swing (#3101). - -## Version 1.6.0-RC3 - -* Fixed the error in 1.6.0-RC2 because of which `Flow.collect` couldn't be called due to the `@InternalCoroutinesApi` annotation (#3082) -* Fixed some R8 warnings introduced in 1.6.0-RC (#3090) -* `TestCoroutineScheduler` now provides a `TimeSource` with its virtual time via the `timeSource` property. Thanks @hfhbd! (#3087) - -## Version 1.6.0-RC2 - -* `@ExperimentalTime` is no longer needed for methods that use `Duration` (#3041). -* `FlowCollector` is now `fun interface`, and corresponding inline extension is removed (#3047). -* Fixed the exception handler being invoked several times on Android, thanks to @1zaman (#3056). -* The deprecated `TestCoroutineScope` is no longer sealed, to simplify migration from it (#3072). -* `runTest` gives more informative errors when it times out waiting for external completion (#3071). -* `SendChannel.trySendBlocking` is now available on Kotlin/Native (#3064). -* Fixed the bug due to which `Dispatchers.Main` was not used for `delay` and `withTimeout` (#3046). -* JDK 1.6 is no longer required for building the project (#3043). -* New version of Dokka is used, fixing the memory leak when building the coroutines and providing brand new reference visuals (https://kotlinlang.org/api/kotlinx.coroutines/) (#3051, #3054). - -## Version 1.6.0-RC - -### kotlinx-coroutines-test rework - -* `kotlinx-coroutines-test` became a multiplatform library usable from K/JVM, K/JS, and K/N. -* Its API was completely reworked to address long-standing issues with consistency, structured concurrency and correctness (#1203, #1609, #2379, #1749, #1204, #1390, #1222, #1395, #1881, #1910, #1772, #1626, #1742, #2082, #2102, #2405, #2462 - ). -* The old API is deprecated for removal, but the new API is based on the similar concepts ([README](kotlinx-coroutines-test/README.md)), and the migration path is designed to be graceful: [migration guide](kotlinx-coroutines-test/MIGRATION.md) - -### Dispatchers - -* Introduced `CoroutineDispatcher.limitedParallelism` that allows obtaining a view of the original dispatcher with limited parallelism (#2919). -* `Dispatchers.IO.limitedParallelism` usages ignore the bound on the parallelism level of `Dispatchers.IO` itself to avoid starvation (#2943). -* Introduced new `Dispatchers.shutdown` method for containerized environments (#2558). -* `newSingleThreadContext` and `newFixedThreadPoolContext` are promoted to delicate API (#2919). - -### Breaking changes +* Coroutines debugger no longer keeps track of coroutines with empty coroutine context (#3782). +* `CopyableThreadContextElement` now properly copies an element when crossing the coroutine boundary in `flowOn` (#3787). Thanks @wanyingd1996! +* Coroutine timeouts no longer prevent K/N `newSingleThreadContext` from closing (#3768). +* A non-linearizability in `Mutex` during `tryLock`/`unlock` sequence with owners is fixed (#3745). +* Atomicfu version is updated to 0.21.0. -* When racing with cancellation, the `future` builder no longer reports unhandled exceptions into the global `CoroutineExceptionHandler`. Thanks @vadimsemenov! (#2774, #2791). -* `Mutex.onLock` is deprecated for removal (#2794). -* `Dispatchers.Main` is now used as the default source of time for `delay` and `withTimeout` when present(#2972). - * To opt-out from this behaviour, `kotlinx.coroutines.main.delay` system property can be set to `false`. -* Java target of coroutines build is now 8 instead of 6 (#1589). +## Version 1.7.1 ### Bug fixes and improvements -* Kotlin is updated to 1.6.0. -* Kotlin/Native [new memory model](https://blog.jetbrains.com/kotlin/2021/08/try-the-new-kotlin-native-memory-manager-development-preview/) is now supported in regular builds of coroutines conditionally depending on whether `kotlin.native.binary.memoryModel` is enabled (#2914). -* Introduced `CopyableThreadContextElement` for mutable context elements shared among multiple coroutines. Thanks @yorickhenning! (#2893). -* `transformWhile`, `awaitClose`, `ProducerScope`, `merge`, `runningFold`, `runingReduce`, and `scan` are promoted to stable API (#2971). -* `SharedFlow.subscriptionCount` no longer conflates incoming updates and gives all subscribers a chance to observe a short-lived subscription (#2488, #2863, #2871). -* `Flow` exception transparency mechanism is improved to be more exception-friendly (#3017, #2860). -* Cancellation from `flat*` operators that leverage multiple coroutines is no longer propagated upstream (#2964). -* `SharedFlow.collect` now returns `Nothing` (#2789, #2502). -* `DisposableHandle` is now `fun interface`, and corresponding inline extension is removed (#2790). -* Deprecation level of all previously deprecated signatures is raised (#3024). -* The version file is shipped with each JAR as a resource (#2941). -* Unhandled exceptions on K/N are passed to the standard library function `processUnhandledException` (#2981). -* A direct executor is used for `Task` callbacks in `kotlinx-coroutines-play-services` (#2990). -* Metadata of coroutines artifacts leverages Gradle platform to have all versions of dependencies aligned (#2865). -* Default `CoroutineExceptionHandler` is loaded eagerly and does not invoke `ServiceLoader` on its exception-handling path (#2552). -* Fixed the R8 rules for `ServiceLoader` optimization (#2880). -* Fixed BlockHound integration false-positives (#2894, #2866, #2937). -* The exception recovery mechanism now uses `ClassValue` when available (#2997). -* JNA is updated to 5.9.0 to support Apple M1 (#3001). -* Obsolete method on internal `Delay` interface is deprecated (#2979). -* Support of deprecated `CommonPool` is removed. - -## Version 1.5.2 - -* Kotlin is updated to 1.5.30. -* New native targets for Apple Silicon are introduced. -* Fixed a bug when `onUndeliveredElement` was incorrectly called on a properly received elements on JS (#2826). -* Fixed `Dispatchers.Default` on React Native, it now fully relies on `setTimeout` instead of stub `process.nextTick`. Thanks to @Legion2 (#2843). -* Optimizations of `Mutex` implementation (#2581). -* `Mutex` implementation is made completely lock-free as stated (#2590). -* Various documentation and guides improvements. Thanks to @MasoodFallahpoor and @Pihanya. - -## Version 1.5.1 - -* Atomic `update`, `getAndUpdate`, and `updateAndGet` operations of `MutableStateFlow` (#2720). -* `Executor.asCoroutineDispatcher` implementation improvements (#2601): - * If the target executor is `ScheduledExecutorService`, then its `schedule` API is used for time-related coroutine operations. - * `RemoveOnCancelPolicy` is now part of the public contract. -* Introduced overloads for `Task.asDeferred` and `Task.await` that accept `CancellationTokenSource` for bidirectional cancellation (#2527). -* Reactive streams are updated to `1.0.3` (#2740). -* `CopyableThrowable` is allowed to modify the exception message during stacktrace recovery (#1931). -* `CoroutineDispatcher.releaseInterceptedContinuation` is now a `final` method (#2785). -* Closing a Handler underlying `Handler.asCoroutineDispatcher` now causes the dispatched coroutines to be canceled on `Dispatchers.IO (#2778)`. -* Kotlin is updated to 1.5.20. -* Fixed a spurious `ClassCastException` in `releaseInterceptedContinuation` and `IllegalStateException` from `tryReleaseClaimedContinuation` (#2736, #2768). -* Fixed inconsistent exception message during stacktrace recovery for non-suspending channel iterators (#2749). -* Fixed linear stack usage for `CompletableFuture.asDeferred` when the target future has a long chain of listeners (#2730). -* Any exceptions from `CoroutineDispatcher.isDispatchNeeded` are now considered as fatal and are propagated to the caller (#2733). -* Internal `DebugProbesKt` (used in the debugger implementation) are moved from `debug` to `core` module. - -## Version 1.5.0 - -Note that this is a full changelog relative to 1.4.3 version. Changelog relative to 1.5.0-RC can be found in the end. - -### Channels API - -* Major channels API rework (#330, #974). Existing `offer`, `poll`, and `sendBlocking` methods are deprecated, internal `receiveCatching` and `onReceiveCatching` removed, `receiveOrNull` and `onReceiveOrNull` are completely deprecated. Previously deprecated `SendChannel.isFull` declaration is removed. Channel operators deprecated with `ERROR` are now `HIDDEN`. -* New methods `receiveCatching`, `onReceiveCatching` `trySend`, `tryReceive`, and `trySendBlocking` along with the new result type `ChannelResult` are introduced. They provide better type safety, are less error-prone, and have a consistent future-proof naming scheme. The full rationale behind this change can be found [here](https://github.com/Kotlin/kotlinx.coroutines/issues/974#issuecomment-806569582). -* `BroadcastChannel` and `ConflatedBroadcastChannel` are marked as `ObsoleteCoroutinesApi` in the favor or `SharedFlow` and `StateFlow`. The migration scheme can be found in their documentation. These classes will be deprecated in the next major release. -* `callbackFlow` and `channelFlow` are promoted to stable API. - -### Reactive integrations - -* All existing API in modules `kotlinx-coroutines-rx2`, `kotlinx-coroutines-rx3`, `kotlinx-coroutines-reactive`, `kotlinx-coroutines-reactor`, and `kotlinx-coroutines-jdk9` were revisited and promoted to stable (#2545). -* `publish` is no longer allowed to emit `null` values (#2646). -* Misleading `awaitSingleOr*` functions on `Publisher` type are deprecated (#2591). -* `MaybeSource.await` is deprecated in the favor of `awaitSingle`, additional lint functions for `Mono` are added in order to prevent ambiguous `Publisher` usages (#2628, #1587). -* `ContextView` support in `kotlinx-coroutines-reactor` (#2575). -* All reactive builders no longer ignore inner cancellation exceptions preventing their completion (#2262, #2646). -* `MaybeSource.collect` and `Maybe.collect` properly finish when they are completed without a value (#2617). -* All exceptions are now consistently handled according to reactive specification, whether they are considered 'fatal' or not by reactive frameworks (#2646). - -### Other improvements - -* Kotlin version is upgraded to 1.5.0 and JVM target is updated to 1.8. -* `Flow.last` and `Flow.lastOrNull` operators (#2246). -* `Flow.runningFold` operator (#2641). -* `CoroutinesTimeout` rule for JUnit5 (#2197). -* Internals of `Job` and `AbstractCoroutine` was reworked, resulting in smaller code size, less memory footprint, and better performance (#2513, #2512). -* `CancellationException` from Kotlin standard library is used for cancellation on Koltin/JS and Kotlin/Native (#2638). -* Introduced new `DelicateCoroutinesApi` annotation that warns users about potential target API pitfalls and suggests studying API's documentation first. The only delicate API right now is `GlobalScope` (#2637). -* Fixed bug introduced in `1.4.3` when `kotlinx-coroutines-core.jar` triggered IDEA debugger failure (#2619). -* Fixed memory leak of `ChildHandlerNode` with reusable continuations (#2564). -* Various documentation improvements (#2555, #2589, #2592, #2583, #2437, #2616, #2633, #2560). - -### Changelog relative to version 1.5.0-RC - -* Fail-fast during `emitAll` called from cancelled `onCompletion` operator (#2700). -* Flows returned by `stateIn`/`shareIn` keep strong reference to sharing job (#2557). -* Rename internal `TimeSource` to `AbstractTimeSource` due to import issues (#2691). -* Reverted the change that triggered IDEA coroutines debugger crash (#2695, reverted #2291). -* `watchosX64` target support for Kotlin/Native (#2524). -* Various documentation fixes and improvements. - -## Version 1.5.0-RC - -### Channels API - -* Major channels API rework (#330, #974). Existing `offer`, `poll`, and `sendBlocking` methods are deprecated, internal `receiveCatching` and `onReceiveCatching` removed, `receiveOrNull` and `onReceiveOrNull` are completely deprecated. Previously deprecated `SendChannel.isFull` declaration is removed. Channel operators deprecated with `ERROR` are now `HIDDEN`. -* New methods `receiveCatching`, `onReceiveCatching` `trySend`, `tryReceive`, and `trySendBlocking` along with the new result type `ChannelResult` are introduced. They provide better type safety, are less error-prone, and have a consistent future-proof naming scheme. The full rationale behind this change can be found [here](https://github.com/Kotlin/kotlinx.coroutines/issues/974#issuecomment-806569582). -* `BroadcastChannel` and `ConflatedBroadcastChannel` are marked as `ObsoleteCoroutinesApi` in the favor or `SharedFlow` and `StateFlow`. The migration scheme can be found in their documentation. These classes will be deprecated in the next major release. -* `callbackFlow` and `channelFlow` are promoted to stable API. - -### Reactive integrations - -* All existing API in modules `kotlinx-coroutines-rx2`, `kotlinx-coroutines-rx3`, `kotlinx-coroutines-reactive`, `kotlinx-coroutines-reactor`, and `kotlinx-coroutines-jdk9` were revisited and promoted to stable (#2545). -* `publish` is no longer allowed to emit `null` values (#2646). -* Misleading `awaitSingleOr*` functions on `Publisher` type are deprecated (#2591). -* `MaybeSource.await` is deprecated in the favor of `awaitSingle`, additional lint functions for `Mono` are added in order to prevent ambiguous `Publisher` usages (#2628, #1587). -* `ContextView` support in `kotlinx-coroutines-reactor` (#2575). -* All reactive builders no longer ignore inner cancellation exceptions preventing their completion (#2262, #2646). -* `MaybeSource.collect` and `Maybe.collect` properly finish when they are completed without a value (#2617). -* All exceptions are now consistently handled according to reactive specification, whether they are considered 'fatal' or not by reactive frameworks (#2646). - -### Other improvements - -* `Flow.last` and `Flow.lastOrNull` operators (#2246). -* `Flow.runningFold` operator (#2641). -* `CoroutinesTimeout` rule for JUnit5 (#2197). -* Internals of `Job` and `AbstractCoroutine` was reworked, resulting in smaller code size, less memory footprint, and better performance (#2513, #2512). -* `CancellationException` from Kotlin standard library is used for cancellation on Koltin/JS and Kotlin/Native (#2638). -* Introduced new `DelicateCoroutineApi` annotation that warns users about potential target API pitfalls and suggests studying API's documentation first. The only delicate API right now is `GlobalScope` (#2637). -* Fixed bug introduced in `1.4.3` when `kotlinx-coroutines-core.jar` triggered IDEA debugger failure (#2619). -* Fixed memory leak of `ChildHandlerNode` with reusable continuations (#2564). -* Various documentation improvements (#2555, #2589, #2592, #2583, #2437, #2616, #2633, #2560). - -## Version 1.4.3 - -### General changes - -* Thread context is properly preserved and restored for coroutines without `ThreadContextElement` (#985) -* `ThreadContextElement`s are now restored in the opposite order from update (#2195) -* Improved performance of combine with 4 parameters, thanks to @alexvanyo (#2419) -* Debug agent sanitizer leaves at least one frame with source location (#1437) -* Update Reactor version in `kotlinx-coroutines-reactor` to `3.4.1`, thanks to @sokomishalov (#2432) -* `callInPlace` contract added to `ReceiveChannel.consume` (#941) -* `CoroutineStart.UNDISPATCHED` promoted to stable API (#1393) -* Kotlin updated to 1.4.30 -* `kotlinx.coroutines` are now released directly to MavenCentral -* Reduced the size of `DispatchedCoroutine` by a field -* Internal class `TimeSource` renamed to `SchedulerTimeSource` to prevent wildcard import issues (#2537) - -### Bug fixes - -* Fixed the problem that prevented implementation via delegation for `Job` interface (#2423) -* Fixed incorrect ProGuard rules that allowed shrinking volatile felds (#1564) -* Fixed `await`/`asDeferred` for `MinimalStage` implementations in jdk8 module (#2456) -* Fixed bug when `onUndeliveredElement` wasn't called for unlimited channels (#2435) -* Fixed a bug when `ListenableFuture.isCancelled` returned from `asListenableFuture` could have thrown an exception, thanks to @vadimsemenov (#2421) -* Coroutine in `callbackFlow` and `produce` is properly cancelled when the channel was closed separately (#2506) - -## Version 1.4.2 - -* Fixed `StackOverflowError` in `Job.toString` when `Job` is observed in its intermediate state (#2371). -* Improved liveness and latency of `Dispatchers.Default` and `Dispatchers.IO` in low-loaded mode (#2381). -* Improved performance of consecutive `Channel.cancel` invocations (#2384). -* `SharingStarted` is now `fun` interface (#2397). -* Additional lint settings for `SharedFlow` to catch programmatic errors early (#2376). -* Fixed bug when mutex and semaphore were not released during cancellation (#2390, thanks to @Tilps for reproducing). -* Some corner cases in cancellation propagation between coroutines and listenable futures are repaired (#1442, thanks to @vadimsemenov). -* Fixed unconditional cast to `CoroutineStackFrame` in exception recovery that triggered failures of instrumented code (#2386). -* Platform-specific dependencies are removed from `kotlinx-coroutines-javafx` (#2360). - -## Version 1.4.1 +* Special characters in coroutine names in JSON dumps are supported (#3747) +* The binary compatibility of the experimental overload of `runTest` is restored (#3673) +* Channels that don't use `onUndeliveredElement` now allocate less memory (#3646) -This is a patch release with an important fix to the `SharedFlow` implementation. +## Version 1.7.0 -* SharedFlow: Fix scenario with concurrent emitters and cancellation of subscriber (#2359, thanks to @vehovsky for the bug report). +### Core API significant improvements -## Version 1.4.0 - -### Improvements - -* `StateFlow`, `SharedFlow` and corresponding operators are promoted to stable API (#2316). -* `Flow.debounce` operator with timeout selector based on each individual element is added (#1216, thanks to @mkano9!). -* `CoroutineContext.job` extension property is introduced (#2159). -* `Flow.combine operator` is reworked: - * Complete fairness is maintained for single-threaded dispatchers. - * Its performance is improved, depending on the use-case, by at least 50% (#2296). - * Quadratic complexity depending on the number of upstream flows is eliminated (#2296). - * `crossinline` and `inline`-heavy internals are removed, fixing sporadic SIGSEGV on Mediatek Android devices (#1683, #1743). -* `Flow.zip` operator performance is improved by 40%. -* Various API has been promoted to stable or its deprecation level has been raised (#2316). - -### Bug fixes - -* Suspendable `stateIn` operator propagates exception to the caller when upstream fails to produce initial value (#2329). -* Fix `SharedFlow` with replay for subscribers working at different speed (#2325). -* Do not fail debug agent installation when security manager does not provide access to system properties (#2311). -* Cancelled lazy coroutines are properly cleaned up from debug agent output (#2294). -* `BlockHound` false-positives are correctly filtered out (#2302, #2190, #2303). -* Potential crash during a race between cancellation and upstream in `Observable.asFlow` is fixed (#2104, #2299, thanks to @LouisCAD and @drinkthestars). - -## Version 1.4.0-M1 +* New `Channel` implementation with significant performance improvements across the API (#3621). +* New `select` operator implementation: faster, more lightweight, and more robust (#3020). +* `Mutex` and `Semaphore` now share the same underlying data structure (#3020). +* `Dispatchers.IO` is added to K/N (#3205) + * `newFixedThreadPool` and `Dispatchers.Default` implementations on K/N were wholly rewritten to support graceful growth under load (#3595). +* `kotlinx-coroutines-test` rework: + - Add the `timeout` parameter to `runTest` for the whole-test timeout, 10 seconds by default (#3270). This replaces the configuration of quiescence timeouts, which is now deprecated (#3603). + - The `withTimeout` exception messages indicate if the timeout used the virtual time (#3588). + - `TestCoroutineScheduler`, `runTest`, and `TestScope` API are promoted to stable (#3622). + - `runTest` now also fails if there were uncaught exceptions in coroutines not inherited from the test coroutine (#1205). ### Breaking changes -* The concept of atomic cancellation in channels is removed. All operations in channels - and corresponding `Flow` operators are cancellable in non-atomic way (#1813). -* If `CoroutineDispatcher` throws `RejectedExecutionException`, cancel current `Job` and schedule its execution to `Dispatchers.IO` (#2003). -* `CancellableContinuation.invokeOnCancellation` is invoked if the continuation was cancelled while its resume has been dispatched (#1915). -* `Flow.singleOrNull` operator is aligned with standard library and does not longer throw `IllegalStateException` on multiple values (#2289). - -### New experimental features - -* `SharedFlow` primitive for managing hot sources of events with support of various subscription mechanisms, replay logs and buffering (#2034). -* `Flow.shareIn` and `Flow.stateIn` operators to transform cold instances of flow to hot `SharedFlow` and `StateFlow` respectively (#2047). - -### Other - -* Support leak-free closeable resources transfer via `onUndeliveredElement` in channels (#1936). -* Changed ABI in reactive integrations for Java interoperability (#2182). -* Fixed ProGuard rules for `kotlinx-coroutines-core` (#2046, #2266). -* Lint settings were added to `Flow` to avoid accidental capturing of outer `CoroutineScope` for cancellation check (#2038). - -### External contributions - -* Allow nullable types in `Flow.firstOrNull` and `Flow.singleOrNull` by @ansman (#2229). -* Add `Publisher.awaitSingleOrDefault|Null|Else` extensions by @sdeleuze (#1993). -* `awaitCancellation` top-level function by @LouisCAD (#2213). -* Significant part of our Gradle build scripts were migrated to `.kts` by @turansky. - -Thank you for your contributions and participation in the Kotlin community! - -## Version 1.3.9 - -* Support of `CoroutineContext` in `Flow.asPublisher` and similar reactive builders (#2155). -* Kotlin updated to 1.4.0. -* Transition to new HMPP publication scheme for multiplatform usages: - * Artifacts `kotlinx-coroutines-core-common` and `kotlinx-coroutines-core-native` are removed. - * For multiplatform usages, it's enough to [depend directly](README.md#multiplatform) on `kotlinx-coroutines-core` in `commonMain` source-set. - * The same artifact coordinates can be used to depend on platform-specific artifact in platform-specific source-set. - -## Version 1.3.8 - -### New experimental features - -* Added `Flow.transformWhile operator` (#2065). -* Replaced `scanReduce` with `runningReduce` to be consistent with the Kotlin standard library (#2139). - -### Bug fixes and improvements - -* Improve user experience for the upcoming coroutines debugger (#2093, #2118, #2131). -* Debugger no longer retains strong references to the running coroutines (#2129). -* Fixed race in `Flow.asPublisher` (#2109). -* Fixed `ensureActive` to work in the empty context case to fix `IllegalStateException` when using flow from `suspend fun main` (#2044). -* Fixed a problem with `AbortFlowException` in the `Flow.first` operator to avoid erroneous `NoSuchElementException` (#2051). -* Fixed JVM dependency on Android annotations (#2075). -* Removed keep rules mentioning `kotlinx.coroutines.android` from core module (#2061 by @mkj-gram). -* Corrected some docs and examples (#2062, #2071, #2076, #2107, #2098, #2127, #2078, #2135). -* Improved the docs and guide on flow cancellation (#2043). -* Updated Gradle version to `6.3` (it only affects multiplatform artifacts in this release). - -## Version 1.3.7 - -* Fixed problem that triggered Android Lint failure (#2004). -* New `Flow.cancellable()` operator for cooperative cancellation (#2026). -* Emissions from `flow` builder now check cancellation status and are properly cancellable (#2026). -* New `currentCoroutineContext` function to use unambiguously in the contexts with `CoroutineScope` in receiver position (#2026). -* `EXACTLY_ONCE` contract support in coroutine builders. -* Various documentation improvements. - -## Version 1.3.6 - -### Flow - -* `StateFlow`, new primitive for state handling (#1973, #1816, #395). The `StateFlow` is designed to eventually replace `ConflatedBroadcastChannel` for state publication scenarios. Please, try it and share your feedback. Note, that Flow-based primitives to publish events will be added later. For events you should continue to either use `BroadcastChannel(1)`, if you put events into the `StateFlow`, protect them from double-processing with flags. -* `Flow.onEmpty` operator is introduced (#1890). -* Behavioural change in `Flow.onCompletion`, it is aligned with `invokeOnCompletion` now and passes `CancellationException` to its cause parameter (#1693). -* A lot of Flow operators have left its experimental status and are promoted to stable API. - -### Other - -* `runInterruptible` primitive to tie cancellation with thread interruption for blocking calls. Contributed by @jxdabc (#1947). -* Integration module with RxJava3 is introduced. Contributed by @ZacSweers (#1883) -* Integration with [BlockHound](https://github.com/reactor/BlockHound) in `kotlinx-coroutines-debug` module (#1821, #1060). -* Memory leak in ArrayBroadcastChannel is fixed (#1885). -* Behavioural change in `suspendCancellableCoroutine`, cancellation is established before invoking passed block argument (#1671). -* Debug agent internals are moved into `kotlinx-coroutines-core` for better integration with IDEA. It should not affect library users and all the redundant code should be properly eliminated with R8. -* ClassCastException with reusable continuations bug is fixed (#1966). -* More precise scheduler detection for `Executor.asCoroutineDispatcher` (#1992). -* Kotlin updated to 1.3.71. - -## Version 1.3.5 - -* `firstOrNull` operator. Contributed by @bradynpoulsen. -* `java.time` adapters for Flow operators. Contributed by @fvasco. -* `kotlin.time.Duration` support (#1402). Contributed by @fvasco. -* Memory leak with a mix of reusable and non-reusable continuations is fixed (#1855). -* `DebugProbes` are ready for production installation: its performance is increased, the flag to disable creation stacktraces to reduce the footprint is introduced (#1379, #1372). -* Stacktrace recovery workaround for Android 6.0 and earlier bug (#1866). -* New integration module: `kotlinx-coroutines-jdk9` with adapters for `java.util.concurrent.Flow`. -* `BroadcastChannel.close` properly starts lazy coroutine (#1713). -* `kotlinx-coroutines-bom` is published without Gradle metadata. -* Make calls to service loader in reactor integrations optimizable by R8 (#1817). - -## Version 1.3.4 - -### Flow - -* Detect missing `awaitClose` calls in `callbackFlow` to make it less error-prone when used with callbacks (#1762, #1770). This change makes `callbackFlow` **different** from `channelFlow`. -* `ReceiveChannel.asFlow` extension is introduced (#1490). -* Enforce exception transparency invariant in `flow` builder (#1657). -* Proper `Dispatcher` support in `Flow` reactive integrations (#1765). -* Batch `Subscription.request` calls in `Flow` reactive integration (#766). -* `ObservableValue.asFlow` added to JavaFx integration module (#1695). -* `ObservableSource.asFlow` added to RxJava2 integration module (#1768). - -### Other changes - -* `kotlinx-coroutines-core` is optimized for R8, making it much smaller for Android usages (75 KB for `1.3.4` release). -* Performance of `Dispatchers.Default` is improved (#1704, #1706). -* Kotlin is updated to 1.3.70. -* `CoroutineDispatcher` and `ExecutorCoroutineDispatcher` experimental coroutine context keys are introduced (#1805). -* Performance of various `Channel` operations is improved (#1565). - -## Version 1.3.3 - -### Flow -* `Flow.take` performance is significantly improved (#1538). -* `Flow.merge` operator (#1491). -* Reactive Flow adapters are promoted to stable API (#1549). -* Reusable cancellable continuations were introduced that improved the performance of various flow operators and iteration over channels (#1534). -* Fixed interaction of multiple flows with `take` operator (#1610). -* Throw `NoSuchElementException` instead of `UnsupportedOperationException` for empty `Flow` in `reduce` operator (#1659). -* `onCompletion` now rethrows downstream exceptions on emit attempt (#1654). -* Allow non-emitting `withContext` from `flow` builder (#1616). - -### Debugging - -* `DebugProbes.dumpCoroutines` is optimized to be able to print the 6-digit number of coroutines (#1535). -* Properly capture unstarted lazy coroutines in debugger (#1544). -* Capture coroutines launched from within a test constructor with `CoroutinesTimeout` test rule (#1542). -* Stacktraces of `Job`-related coroutine machinery are shortened and prettified (#1574). -* Stacktrace recovery unification that should provide a consistent experience recover of stacktrace (#1597). -* Stacktrace recovery for `withTimeout` is supported (#1625). -* Do not recover exception with a single `String` parameter constructor that is not a `message` (#1631). - -### Other features - -* `Dispatchers.Default` and `Dispatchers.IO` rework: CPU consumption is significantly lower, predictable idle threads termination (#840, #1046, #1286). -* Avoid `ServiceLoader` for loading `Dispatchers.Main` (#1572, #1557, #878, #1606). -* Consistently handle undeliverable exceptions in RxJava and Reactor integrations (#252, #1614). -* `yield` support in immediate dispatchers (#1474). -* `CompletableDeferred.completeWith(result: Result<T>)` is introduced. -* Added support for tvOS and watchOS-based Native targets (#1596). +* Old K/N memory model is no longer supported (#3375). +* New generic upper bounds were added to reactive integration API where the language since 1.8.0 dictates (#3393). +* `kotlinx-coroutines-core` and `kotlinx-coroutines-jdk8` artifacts were merged into a single artifact (#3268). +* Artificial stackframes in stacktrace recovery no longer contain the `\b` symbol and are now navigable in IDE and supplied with proper documentation (#2291). +* `CoroutineContext.isActive` returns `true` for contexts without any job in them (#3300). ### Bug fixes and improvements -* Kotlin version is updated to 1.3.61. -* `CoroutineDispatcher.isDispatchNeeded` is promoted to stable API (#1014). -* Livelock and stackoverflows in mutual `select` expressions are fixed (#1411, #504). -* Properly handle `null` values in `ListenableFuture` integration (#1510). -* Making ReceiveChannel.cancel linearizability-friendly. -* Linearizability of Channel.close in a complex contended cases (#1419). -* ArrayChannel.isBufferEmpty atomicity is fixed (#1526). -* Various documentation improvements. -* Reduced bytecode size of `kotlinx-coroutines-core`, reduced size of minified `dex` when using basic functionality of `kotlinx-coroutines`. - -## Version 1.3.2 - -This is a maintenance release that does not include any new features or bug fixes. - -* Reactive integrations for `Flow` are promoted to stable API. -* Obsolete reactive API is deprecated. -* Deprecation level for API deprecated in 1.3.0 is increased. -* Various documentation improvements. - -## Version 1.3.1 - -This is a minor update with various fixes: -* Flow: Fix recursion in combineTransform<T1, T2, R> (#1466). -* Fixed race in the Semaphore (#1477). -* Repaired some of ListenableFuture.kt's cancellation corner cases (#1441). -* Consistently unwrap exception in slow path of CompletionStage.asDeferred (#1479). -* Various fixes in documentation (#1496, #1476, #1470, #1468). -* Various cleanups and additions in tests. - -Note: Kotlin/Native artifacts are now published with Gradle metadata format version 1.0, so you will need -Gradle version 5.3 or later to use this version of kotlinx.coroutines in your Kotlin/Native project. - -## Version 1.3.0 - -### Flow - -This version is the first stable release with [`Flow`](https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.flow/-flow/index.html) API. - -All `Flow` API not marked with `@FlowPreview` or `@ExperimentalCoroutinesApi` annotations are stable and here to stay. -Flow declarations marked with `@ExperimentalCoroutinesApi` have [the same guarantees](/docs/topics/compatibility.md#experimental-api) as regular experimental API. -Please note that API marked with `@FlowPreview` have [weak guarantees](/docs/topics/compatibility.md#flow-preview-api) on source, binary and semantic compatibility. - -### Changelog - -* A new [guide section](/docs/topics/flow.md) about Flow. -* `CoroutineDispatcher.asExecutor` extension (#1450). -* Fixed bug when `select` statement could report the same exception twice (#1433). -* Fixed context preservation in `flatMapMerge` in a case when collected values were immediately emitted to another flow (#1440). -* Reactive Flow integrations enclosing files are renamed for better interoperability with Java. -* Default buffer size in all Flow operators is increased to 64. -* Kotlin updated to 1.3.50. - -## Version 1.3.0-RC2 - -### Flow improvements -* Operators for UI programming are reworked for the sake of consistency, naming scheme for operator overloads is introduced: - * `combineLatest` is deprecated in the favor of `combine`. - * `combineTransform` operator for non-trivial transformations (#1224). - * Top-level `combine` and `combineTransform` overloads for multiple flows (#1262). - * `switchMap` is deprecated. `flatMapLatest`, `mapLatest` and `transformLatest` are introduced instead (#1335). - * `collectLatest` terminal operator (#1269). - -* Improved cancellation support in `flattenMerge` (#1392). -* `channelFlow` cancellation does not leak to the parent (#1334). -* Fixed flow invariant enforcement for `suspend fun main` (#1421). -* `delayEach` and `delayFlow` are deprecated (#1429). - -### General changes -* Integration with Reactor context - * Propagation of the coroutine context of `await` calls into Mono/Flux builder. - * Publisher.asFlow propagates coroutine context from `collect` call to the Publisher. - * New `Flow.asFlux ` builder. - -* ServiceLoader-code is adjusted to avoid I/O on the Main thread on newer (3.6.0+) Android toolchain. -* Stacktrace recovery support for minified builds on Android (#1416). -* Guava version in `kotlinx-coroutines-guava` updated to `28.0`. -* `setTimeout`-based JS dispatcher for platforms where `process` is unavailable (#1404). -* Native, JS and common modules are added to `kotlinx-coroutines-bom`. -* Fixed bug with ignored `acquiredPermits` in `Semaphore` (#1423). - -## Version 1.3.0-RC - -### Flow - -* Core `Flow` API is promoted to stable -* New basic `Flow` operators: `withIndex`, `collectIndexed`, `distinctUntilChanged` overload -* New core `Flow` operators: `onStart` and `onCompletion` -* `ReceiveChannel.consumeAsFlow` and `emitAll` (#1340) - -### General changes - -* Kotlin updated to 1.3.41 -* Added `kotlinx-coroutines-bom` with Maven Bill of Materials (#1110) -* Reactive integrations are seriously improved - * All builders now are top-level functions instead of extensions on `CoroutineScope` and prohibit `Job` instance in their context to simplify lifecycle management - * Fatal exceptions are handled consistently (#1297) - * Integration with Reactor Context added (#284) -* Stacktrace recovery for `suspend fun main` (#1328) -* `CoroutineScope.cancel` extension with message (#1338) -* Protection against non-monotonic clocks in `delay` (#1312) -* `Duration.ZERO` is handled properly in JDK 8 extensions (#1349) -* Library code is adjusted to be more minification-friendly - -## Version 1.3.0-M2 - - * Kotlin updated to 1.3.40. - * `Flow` exception transparency concept. - * New declarative `Flow` operators: `onCompletion`, `catch`, `retryWhen`, `launchIn`. `onError*` operators are deprecated in favour of `catch`. (#1263) - * `Publisher.asFlow` is integrated with `buffer` operator. - * `Publisher.openSubscription` default request size is `1` instead of `0` (#1267). - -## Version 1.3.0-M1 - -Flow: - * Core `Flow` interfaces and operators are graduated from preview status to experimental. - * Context preservation invariant rework (#1210). - * `channelFlow` and `callbackFlow` replacements for `flowViaChannel` for concurrent flows or callback-based APIs. - * `flow` prohibits emissions from non-scoped coroutines by default and recommends to use `channelFlow` instead to avoid most of the concurrency-related bugs. - * Flow cannot be implemented directly - * `AbstractFlow` is introduced for extension (e.g. for managing state) and ensures all context preservation invariants. - * Buffer size is decoupled from all operators that imply channel usage (#1233) - * `buffer` operator can be used to adjust buffer size of any buffer-dependent operator (e.g. `channelFlow`, `flowOn` and `flatMapMerge`). - * `conflate` operator is introduced. - * Flow performance is significantly improved. - * New operators: `scan`, `scanReduce`, `first`, `emitAll`. - * `flowWith` and `flowViaChannel` are deprecated. - * `retry` ignores cancellation exceptions from upstream when the flow was externally cancelled (#1122). - * `combineLatest` overloads for multiple flows (#1193). - * Fixed numerical overflow in `drop` operator. - -Channels: - * `consumeEach` is promoted to experimental API (#1080). - * Conflated channels always deliver the latest value after closing (#332, #1235). - * Non-suspending `ChannelIterator.next` to improve iteration performance (#1162). - * Channel exception types are consistent with `produce` and are no longer swallowed as cancellation exceptions in case of programmatic errors (#957, #1128). - * All operators on channels (that were prone to coroutine leaks) are deprecated in the favor of `Flow`. - -General changes: - * Kotlin updated to 1.3.31 - * `Semaphore` implementation (#1088) - * Loading of `Dispatchers.Main` is tweaked so the latest version of R8 can completely remove I/O when loading it (#1231). - * Performace of all JS dispatchers is significantly improved (#820). - * `withContext` checks cancellation status on exit to make reasoning about sequential concurrent code easier (#1177). - * Consistent exception handling mechanism for complex hierarchies (#689). - * Convenient overload for `CoroutinesTimeout.seconds` (#1184). - * Fix cancellation bug in onJoin (#1130). - * Prevent internal names clash that caused errors for ProGuard (#1159). - * POSIX's `nanosleep` as `delay` in `runBlocking ` in K/N (#1225). - -## Version 1.2.2 - -* Kotlin updated to 1.3.40. - -## Version 1.2.1 - -Major: - * Infrastructure for testing coroutine-specific code in `kotlinx-coroutines-test`: `runBlockingTest`, `TestCoroutineScope` and `TestCoroutineDispatcher`, contributed by Sean McQuillan (@objcode). Obsolete `TestCoroutineContext` from `kotlinx-coroutines-core` is deprecated. - * `Job.asCompletableFuture` extension in jdk8 module (#1113). - -Flow improvements: - * `flowViaChannel` rework: block parameter is no longer suspending, but provides `CoroutineScope` receiver and allows conflated channel (#1081, #1112). - * New operators: `switchMap`, `sample`, `debounce` (#1107). - * `consumerEach` is deprecated on `Publisher`, `ObservableSource` and `MaybeSource`, `collect` extension is introduced instead (#1080). - -Other: - * Race in Job.join and concurrent cancellation is fixed (#1123). - * Stacktrace recovery machinery improved: cycle detection works through recovered exceptions, cancellation exceptions are recovered on cancellation fast-path. - * Atomicfu-related bug fixes: publish transformed artifacts, do not propagate transitive atomicfu dependency (#1064, #1116). - * Publication to NPM fixed (#1118). - * Misplaced resources are removed from the final jar (#1131). - -## Version 1.2.0 - - * Kotlin updated to 1.3.30. - * New API: `CancellableContinuation.resume` with `onCancelling` lambda (#1044) to consistently handle closeable resources. - * Play services task version updated to 16.0.1. - * `ReceiveChannel.isEmpty` is no longer deprecated - -A lot of `Flow` improvements: - * Purity property is renamed to context preservation and became more restrictive. - * `zip` and `combineLatest` operators. - * Integration with RxJava2 - * `flatMap`, `merge` and `concatenate` are replaced with `flattenConcat`, `flattenMerge`, `flatMapConcat` and `flatMapMerge`. - * Various documentation improvements and minor bug fixes. - -Note that `Flow` **is not** leaving its [preview status](/docs/topics/compatibility.md#flow-preview-api). - -## Version 1.2.0-alpha-2 - -This release contains major [feature preview](/docs/topics/compatibility.md#flow-preview-api): cold streams aka `Flow` (#254). - -Performance: -* Performance of `Dispatcher.Main` initialization is significantly improved (#878). - -## Version 1.2.0-alpha - -* Major debug agent improvements. Real stacktraces are merged with coroutine stacktraces for running coroutines, merging heuristic is improved, API is cleaned up and is on its road to stabilization (#997). -* `CoroutineTimeout` rule or JUnit4 is introduced to simplify coroutines debugging (#938). -* Stacktrace recovery improvements. Exceptions with custom properties are no longer copied, `CopyableThrowable` interface is introduced, machinery is [documented](https://github.com/Kotlin/kotlinx.coroutines/blob/develop/docs/debugging.md) (#921, #950). -* `Dispatchers.Unconfined`, `MainCoroutineDispatcher.immediate`, `MainScope` and `CoroutineScope.cancel` are promoted to stable API (#972). -* `CompletableJob` is introduced (#971). -* Structured concurrency is integrated into futures and listenable futures (#1008). -* `ensurePresent` and `isPresent` extensions for `ThreadLocal` (#1028). -* `ensureActive` extensions for `CoroutineContext`, `CoroutineScope` and `Job` (#963). -* `SendChannel.isFull` and `ReceiveChannel.isEmpty` are deprecated (#1053). -* `withContext` checks cancellation on entering (#962). -* Operator `invoke` on `CoroutineDispatcher` (#428). -* Java 8 extensions for `delay` and `withTimeout` now properly handle too large values (#428). -* A global exception handler for fatal exceptions in coroutines is introduced (#808, #773). -* Major improvements in cancellation machinery and exceptions delivery consistency. Cancel with custom exception is completely removed. -* Kotlin version is updated to 1.3.21. -* Do not use private API on newer Androids to handle exceptions (#822). - -Bug fixes: -* Proper `select` support in debug agent (#931). -* Proper `supervisorScope` support in debug agent (#915). -* Throwing `initCause` does no longer trigger an internal error (#933). -* Lazy actors are started when calling `close` in order to cleanup their resources (#939). -* Minor bugs in reactive integrations are fixed (#1008). -* Experimental scheduler shutdown sequence is fixed (#990). - -## Version 1.1.1 - -* Maintenance release, no changes in the codebase -* Kotlin is updated to 1.3.20 -* Gradle is updated to 4.10 -* Native module is published with Gradle metadata v0.4 - -## Version 1.1.0 - -* Kotlin version updated to 1.3.11. -* Resumes to `CancellableContinuation` in the final state produce `IllegalStateException` (#901). This change does not affect #830, races between resume and cancellation do not lead to an exceptional situation. -* `runBlocking` is integrated with `Dispatchers.Unconfined` by sharing an internal event loop. This change does not affect the semantics of the previously correct code but allows to mix multiple `runBlocking` and unconfined tasks (#860). - -## Version 1.1.0-alpha - -### Major improvements in coroutines testing and debugging -* New module: [`kotlinx-coroutines-debug`](https://github.com/Kotlin/kotlinx.coroutines/blob/master/core/kotlinx-coroutines-debug/README.md). Debug agent that improves coroutines stacktraces, allows to print all active coroutines and its hierarchies and can be installed as Java agent. -* New module: [`kotlinx-coroutines-test`](https://github.com/Kotlin/kotlinx.coroutines/blob/master/core/kotlinx-coroutines-test/README.md). Allows setting arbitrary `Dispatchers.Main` implementation for tests (#810). -* Stacktrace recovery mechanism. Exceptions from coroutines are recovered from current coroutine stacktraces to simplify exception diagnostic. Enabled in debug mode, controlled by `kotlinx.coroutines.debug` system property (#493). - -### Other improvements -* `MainScope` factory and `CoroutineScope.cancel` extension (#829). One line `CoroutineScope` integration! -* `CancellableContinuation` race between `resumeWithException` and `cancel` is addressed, exceptions during cancellation are no longer reported to exception handler (#830, #892). -* `Dispatchers.Default` now consumes much less CPU on JVM (#840). -* Better diagnostic and fast failure if an uninitialized dispatcher is used (#880). -* Conflated channel becomes linearizable. -* Fixed inconsistent coroutines state when the result of the coroutine had type `DisposableHandle` (#835). -* Fixed `JavaFx` initialization bug (#816). -* `TimeoutCancellationException` is thrown by `withTimeout` instead of `CancellationException` if negative timeout is supplied (#870). -* Kotlin/Native single-threaded workers support: coroutines can be safely used in multiple independent K/N workers. -* jsdom support in `Dispatchers.Default` on JS. -* rxFlowable generic parameter is now restricted with Any. -* Guava 27 support in `kotlinx-coroutines-guava`. -* Coroutines are now built with progressive mode. -* Various fixes in the documentation. - -## Version 1.0.1 - -* Align `publisher` implementation with Reactive TCK. -* Reimplement `future` coroutine builders on top of `AbstractCoroutine` (#751). -* Performance optimizations in `Dispatchers.Default` and `Dispatchers.IO`. -* Use only public API during `JavaFx` instantiation, fixes warnings on Java 9 and build on Java 11 (#463). -* Updated contract of `CancellableContinuation.resumeWithException` (documentation fix, see #712). -* Check cancellation on fast-path of all in-place coroutine builders (`withContext`, `coroutineScope`, `supervisorScope`, `withTimeout` and `withTimeoutOrNull`). -* Add optional prefix to thread names of `ExperimentalCoroutineDispatcher` (#661). -* Fixed bug when `ExperimentalCoroutineDispatcher` could end up in inconsistent state if `Thread` constructor throws an exception (#748). - -## Version 1.0.0 - -* All Kotlin dependencies updated to 1.3 release version. -* Fixed potential memory leak in `HandlerDispatcher.scheduleResumeAfterDelay`, thanks @cbeyls. -* `yield` support for `Unconfined` and immediate dispatchers (#737). -* Various documentation improvements. - -## Version 1.0.0-RC1 - -* Coroutines API is updated to Kotlin 1.3. -* Deprecated API is removed or marked as `internal`. -* Experimental and internal coroutine API is marked with corresponding `kotlin.experimental.Experimental` annotation. If you are using `@ExperimentalCoroutinesApi` or `@InternalCoroutinesApi` you should explicitly opt-in, otherwise compilation warning (or error) will be produced. -* `Unconfined` dispatcher (and all dispatchers which support immediate invocation) forms event-loop on top of current thread, thus preventing all `StackOverflowError`s. `Unconfined` dispatcher is now much safer for the general use and may leave its experimental status soon (#704). -* Significantly improved performance of suspending hot loops in `kotlinx.coroutines` (#537). -* Proguard rules are embedded into coroutines JAR to assist jettifier (#657) -* Fixed bug in shutdown sequence of `runBlocking` (#692). -* `ReceiveChannel.receiveOrNull` is marked as obsolete and deprecated. -* `Job.cancel(cause)` and `ReceiveChannel.cancel(cause)` are deprecated, `cancel()` returns `Unit` (#713). - -## Version 0.30.2 - -* `Dispatchers.Main` is instantiated lazily (see #658 and #665). -* Blocking coroutine dispatcher views are now shutdown properly (#678). -* Prevent leaking Kotlin 1.3 from atomicfu dependency (#659). -* Thread-pool based dispatcher factories are marked as obsolete (#261). -* Fixed exception loss on `withContext` cancellation (#675). - -## Version 0.30.1 - -Maintenance release: -* Added `Dispatchers.Main` to common dispatchers, which can be used from Android, Swing and JavaFx projects if a corresponding integration library is added to dependencies. -* With `Dispatchers.Main` improvement tooling bug in Android Studio #626 is mitigated, so Android users now can safely start the migration to the latest `kotlinx.coroutines` version. -* Fixed bug with thread unsafety of shutdown sequence in `EventLoop`. -* Experimental coroutine dispatcher now has `close` contract similar to Java `Executor`, so it can be safely instantiated and closed multiple times (affects only unit tests). -* Atomicfu version is updated with fixes in JS transformer (see #609) - -## Version 0.30.0 - -* **[Major]** Further improvements in exception handling — no failure exception is lost. - * `async` and async-like builders cancel parent on failure (it affects `CompletableDeferred`, and all reactive integration builders). - * This makes parallel decomposition exception-safe and reliable without having to rember about `awaitAll` (see #552). - * `Job()` wih parent now also cancels parent on failure consistently with other scopes. - * All coroutine builders and `Job` implementations propagate failure to the parent unless it is a `CancellationException`. - * Note, "scoping" builders don't "cancel the parent" verbatim, but rethrow the corresponding exception to the caller for handling. - * `SupervisorJob()` and `supervisorScope { ... }` are introduced, allowing for a flexible implementation of custom exception-handling policies, see a [new section in the guide on supervision](docs/topics/exception-handling.md#supervision). - * Got rid of `awaitAll` in documentation and rewrote `currentScope` section (see #624). -* **[Major]** Coroutine scheduler is used for `Dispatchers.Default` by default instead of deprecated `CommonPool`. - * "`DefaultDispatcher`" is used as a public name of the default impl (you'll see it thread names and in the guide). - * `-Dkotlinx.coroutines.scheduler=off` can be used to switch back to `CommonPool` for a time being (until deprecated CommonPool is removed). -* Make `CoroutineStart.ATOMIC` experimental as it covers important use-case with resource cleanup in finally block (see #627). -* Restored binary compatibility of `Executor.asCoroutineDispatcher` (see #629). -* Fixed OOM in thread-pool dispatchers (see #571). -* Check for cancellation when starting coroutine with `Dispatchers.Unconfined` (see #621). -* A bunch of various performance optimizations and docs fixes, including contributions from @AlexanderPrendota, @PaulWoitaschek. - -## Version 0.27.0 - -* **[Major]** Public API revision. All public API was reviewed and marked as preparation to `1.0` release: - 1. `@Deprecated` API. All API marked as deprecated will be removed in 1.0 release without replacement. - 2. `@ExperimentalCoroutinesApi` API. This API is experimental and may change in the future, but migration mechanisms will be provided. Signature, binary compatibility and semantics can be changed. - 3. `@InternalCoroutinesApi`. This API is intended to be used **only** from within `kotlinx.coroutines`. It can and will be changed, broken - and removed in the future releases without any warnings and migration aids. If you find yourself using this API, it is better to report - your use-case to Github issues, so decent, stable and well-tested alternative can be provided. - 4. `@ObsoleteCoroutinesApi`. This API has serious known flaws and will be replaced with a better alternative in the nearest releases. - 5. Regular public API. This API is proven to be stable and is not going to be changed. If at some point it will be discovered that such API - has unfixable design flaws, it will be gradually deprecated with proper replacement and migration aid, but won't be removed for at least a year. -* **[Major]** Job state machine is reworked. It includes various performance improvements, fixes in -data-races which could appear in a rare circumstances and consolidation of cancellation and exception handling. -Visible consequences of include more robust exception handling for large coroutines hierarchies and for different kinds of `CancellationException`, transparent parallel decomposition and consistent view of coroutines hierarchy in terms of its state (see #220 and #585). -* NIO, Quasar and Rx1 integration modules are removed with no replacement (see #595, #601, #603). -* `withContext` is now aligned with structured concurrency and awaits for all launched tasks, its performance is significantly improved (see #553 and #617). -* Added integration module with Play Services Task API. Thanks @SUPERCILEX and @lucasvalenteds for the contribution! -* Integration with Rx2 now respects nullability in type constraints (see #347). Thanks @Dmitry-Borodin for the contribution! -* `CompletableFuture.await` and `ListenableFuture.await` now propagate cancellation to the future (see #611). -* Cancellation of `runBlocking` machinery is improved (see #589). -* Coroutine guide is restructured and split to multiple files for the sake of simplicity. -* `CoroutineScope` factory methods add `Job` if it is missing from the context to enforce structured concurrency (see #610). -* `Handler.asCoroutineDispatcher` has a `name` parameter for better debugging (see #615). -* Fixed bug when `CoroutineSchedule` was closed from one of its threads (see #612). -* Exceptions from `CoroutineExceptionHandler` are reported by default exception handler (see #562). -* `CoroutineName` is now available from common modules (see #570). -* Update to Kotlin 1.2.70. - -## Version 0.26.1 - -* Android `Main` dispatcher is `async` by default which may significantly improve UI performance. Contributed by @JakeWharton (see #427). -* Fixed bug when lazily-started coroutine with registered cancellation handler was concurrently started and cancelled. -* Improved termination sequence in IO dispatcher. -* Fixed bug with `CoroutineScope.plus` operator (see #559). -* Various fixes in the documentation. Thanks to @SUPERCILEX, @yorlov, @dualscyther and @soudmaijer! - -## Version 0.26.0 - -* Major rework of `kotlinx.coroutines` concurrency model (see #410 for a full explanation of the rationale behind this change): - * All coroutine builders are now extensions on `CoroutineScope` and inherit its `coroutineContext`. Standalone builders are deprecated. - * As a consequence, all nested coroutines launched via builders now automatically establish parent-child relationship and inherit `CoroutineDispatcher`. - * All coroutine builders use `Dispatchers.Default` by default if `CoroutineInterceptor` is not present in their context. - * [CoroutineScope](https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/-coroutine-scope/index.html) became the first-class citizen in `kolinx.coroutines`. - * `withContext` `block` argument has `CoroutineScope` as a receiver. - * [GlobalScope](https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/-global-scope/index.html) is introduced to simplify migration to new API and to launch global-level coroutines. - * `currentScope` and `coroutineScope` builders are introduced to extract and provide `CoroutineScope`. - * Factory methods to create `CoroutineScope` from `CoroutineContext` are introduced. - * `CoroutineScope.isActive` became an extension property. - * New sections about structured concurrency in core guide: ["Structured concurrency"](docs/topics/coroutines-guide.md#structured-concurrency), ["Scope builder"](docs/topics/coroutines-guide.md#scope-builder) and ["Structured concurrency with async"](docs/topics/coroutines-guide.md#structured-concurrency-with-async). - * New section in UI guide with Android example: ["Structured concurrency, lifecycle and coroutine parent-child hierarchy"](ui/coroutines-guide-ui.md#structured-concurrency,-lifecycle-and-coroutine-parent-child-hierarchy). - * Deprecated reactive API is removed. -* Dispatchers are renamed and grouped in the Dispatchers object (see #41 and #533): - * Dispatcher names are consistent. - * Old dispatchers including `CommonPool` are deprecated. -* Fixed bug with JS error in rare cases in `invokeOnCompletion(onCancelling = true)`. -* Fixed loading of Android exception handler when `Thread.contextClassLoader` is mocked (see #530). -* Fixed bug when `IO` dispatcher silently hung (see #524 and #525) . - -## Version 0.25.3 - -* Distribution no longer uses multi-version jar which is not supported on Android (see #510). -* JS version of the library does not depend on AtomicFu anymore: - All the atomic boxes in JS are fully erased. -* Note that versions 0.25.1-2 are skipped for technical reasons (they were not fully released). - -## Version 0.25.0 - -* Major rework on exception-handling and cancellation in coroutines (see #333, #452 and #451): - * New ["Exception Handling" section in the guide](docs/topics/coroutines-guide.md#exception-handling) explains exceptions in coroutines. - * Semantics of `Job.cancel` resulting `Boolean` value changed — `true` means exception was handled by the job, caller shall handle otherwise. - * Exceptions are properly propagated from children to parents. - * Installed `CoroutineExceptionHandler` for a family of coroutines receives one aggregated exception in case of failure. - * Change `handleCoroutineException` contract, so custom exception handlers can't break coroutines machinery. - * Unwrap `JobCancellationException` properly to provide exception transparency over whole call chain. -* Introduced support for thread-local elements in coroutines context (see #119): - * `ThreadContextElement` API for custom thread-context sensitive context elements. - * `ThreadLocal.asContextElement()` extension function to convert an arbitrary thread-local into coroutine context element. - * New ["Thread-local data" subsection in the guide](docs/topics/coroutines-guide.md#thread-local-data) with examples. - * SLF4J Mapped Diagnostic Context (MDC) integration is provided via `MDCContext` element defined in [`kotlinx-coroutines-slf4j`](integration/kotlinx-coroutines-slf4j/README.md) integration module. -* Introduced IO dispatcher to offload blocking I/O-intensive tasks (see #79). -* Introduced `ExecutorCoroutineDispatcher` instead of `CloseableCoroutineDispatcher` (see #385). -* Built with Kotlin 1.2.61 and Kotlin/Native 0.8.2. -* JAR files for `kotlinx-coroutines` are now [JEP 238](https://openjdk.java.net/jeps/238) multi-release JAR files. - * On JDK9+ `VarHandle` is used for atomic operations instead of `Atomic*FieldUpdater` for better performance. - * See [AtomicFu](https://github.com/Kotlin/kotlinx.atomicfu/blob/master/README.md) project for details. -* Reversed addition of `BlockingChecker` extension point to control where `runBlocking` can be used (see #227). - * `runBlocking` can be used anywhere without limitations (again), but it would still cause problems if improperly used on UI thread. -* Corrected return-type of `EventLoop` pseudo-constructor (see #477, PR by @Groostav). -* Fixed `as*Future()` integration functions to catch all `Throwable` exceptions (see #469). -* Fixed `runBlocking` cancellation (see #501). -* Fixed races and timing bugs in `withTimeoutOrNull` (see #498). -* Execute `EventLoop.invokeOnTimeout` in `DefaultDispatcher` to allow busy-wait loops inside `runBlocking` (see #479). -* Removed `kotlinx-coroutines-io` module from the project, it has moved to [kotlinx-io](https://github.com/kotlin/kotlinx-io/). -* Provide experimental API to create limited view of experimental dispatcher (see #475). -* Various minor fixes by @LouisCAD, @Dmitry-Borodin. - -## Version 0.24.0 - -* Fully multiplatform release with Kotlin/Native support (see #246): - * Only single-threaded operation inside `runBlocking` event loop is supported at this moment. - * See details on setting up build environment [here](native/README.md). -* Improved channels: - * Introduced `SendChannel.invokeOnClose` (see #341). - * Make `close`, `cancel`, `isClosedForSend`, `isClosedForReceive` and `offer` linearizable with other operations (see #359). - * Fixed bug when send operation can be stuck in channel forever. - * Fixed broadcast channels on JS (see #412). -* Provides `BlockingChecker` mechanism which checks current context (see #227). - * Attempts to use `runBlocking` from any supported UI thread (Android, JavaFx, Swing) will result in exception. -* Android: - * Worked around Android bugs with zero-size ForkJoinPool initialization (see #432, #288). - * Introduced `UI.immediate` extension as performance-optimization to immediately execute tasks which are invoked from the UI thread (see #381). - * Use it only when absolutely needed. It breaks asynchrony of coroutines and may lead to surprising and unexpected results. -* Fixed materialization of a `cause` exception for `Job` onCancelling handlers (see #436). -* Fixed JavaFx `UI` on Java 9 (see #443). -* Fixed and documented the order between cancellation handlers and continuation resume (see #415). -* Fixed resumption of cancelled continuation (see #450). -* Includes multiple fixes to documentation contributed by @paolop, @SahilLone, @rocketraman, @bdavisx, @mtopolnik, @Groostav. -* Experimental coroutines scheduler preview (JVM only): - * Written from scratch and optimized for communicating coroutines. - * Performs significantly better than ForkJoinPool on coroutine benchmarks and for connected applications with [ktor](https://ktor.io). - * Supports automatic creating of new threads for blocking operations running on the same thread pool (with an eye on solving #79), but there is no stable public API for it just yet. - * For preview, run JVM with `-Dkotlinx.coroutines.scheduler` option. In this case `DefaultDispatcher` is set to new experimental scheduler instead of FJP-based `CommonPool`. - * Submit your feedback to issue #261. - -## Version 0.23.4 - -* Recompiled with Kotlin 1.2.51 to solve broken metadata problem (see [KT-24944](https://youtrack.jetbrains.com/issue/KT-24944)). - -## Version 0.23.3 - -* Kotlin 1.2.50. -* JS: Moved to atomicfu version 0.10.3 that properly matches NPM & Kotlin/JS module names (see #396). -* Improve source-code compatibility with previous (0.22.x) version of `openChannel().use { ... }` pattern by providing deprecated extension function `use` on `ReceiveChannel`. - -## Version 0.23.2 - -* IO: fix joining and continuous writing byte array interference. - -## Version 0.23.1 - -* JS: Fix dependencies in NPM: add "kotlinx-atomicfu" dependency (see #370). -* Introduce `broadcast` coroutine builder (see #280): - * Support `BroadcastChannel.cancel` method to drop the buffer. - * Introduce `ReceiveChannel.broadcast()` extension. -* Fixed a bunch of doc typos (PRs by @paolop). -* Corrected previous version's release notes (PR by @ansman). - -## Version 0.23.0 - -* Kotlin 1.2.41 -* **Coroutines core module is made mostly cross-platform for JVM and JS**: - * Migrate channels and related operators to common, so channels can be used from JS (see #201). - * Most of the code is shared between JVM and JS versions using cross-platform version of [AtomicFU](https://github.com/Kotlin/kotlinx.atomicfu) library. - * The recent version of Kotlin allows default parameters in common code (see #348). - * The project is built using Gradle 4.6. -* **Breaking change**: `CancellableContinuation` is not a `Job` anymore (see #219): - * It does not affect casual users of `suspendCancellableCoroutine`, since all the typically used functions are still there. - * `CancellableContinuation.invokeOnCompletion` is deprecated now and its semantics had subtly changed: - * `invokeOnCancellation` is a replacement for `invokeOnCompletion` to install a handler. - * The handler is **not** invoked on `resume` which corresponds to the typical usage pattern. - * There is no need to check for `cont.isCancelled` in a typical handler code anymore (since handler is invoked only when continuation is cancelled). - * Multiple cancellation handlers cannot be installed. - * Cancellation handlers cannot be removed (disposed of) anymore. - * This change is designed to allow better performance of suspending cancellable functions: - * Now `CancellableContinuation` implementation has simpler state machine and is implemented more efficiently. - * Exception handling in `AbstractContinuation` (that implements `CancellableContinuation`) is now consistent: - * Always prefer exception thrown from coroutine as exceptional reason, add cancellation cause as suppressed exception. -* **Big change**: Deprecate `CoroutineScope.coroutineContext`: - * It is replaced with top-level `coroutineContext` function from Kotlin standard library. -* Improve `ReceiveChannel` operators implementations to guarantee closing of the source channels under all circumstances (see #279): - * `onCompletion` parameter added to `produce` and all other coroutine builders. - * Introduce `ReceiveChannel.consumes(): CompletionHandler` extension function. -* Replace `SubscriptionReceiveChannel` with `ReceiveChannel` (see #283, PR by @deva666). - * `ReceiveChannel.use` extension is introduced to preserve source compatibility, but is deprecated. - * `consume` or `consumeEach` extensions should be used for channels. - * When writing operators, `produce(onCompletion=consumes()) { ... }` pattern shall be used (see #279 above). -* JS: Kotlin is declared as peer dependency (see #339, #340, PR by @ansman). -* Invoke exception handler for actor on cancellation even when channel was successfully closed, so exceptions thrown by actor are always reported (see #368). -* Introduce `awaitAll` and `joinAll` for `Deferred` and `Job` lists correspondingly (see #171). -* Unwrap `CompletionException` exception in `CompletionStage.await` slow-path to provide consistent results (see #375). -* Add extension to `ExecutorService` to return `CloseableCoroutineDispatcher` (see #278, PR by @deva666). -* Fail with proper message during build if JDK_16 is not set (see #291, PR by @venkatperi). -* Allow negative timeouts in `delay`, `withTimeout` and `onTimeout` (see #310). -* Fix a few bugs (leaks on cancellation) in `delay`: - * Invoke `clearTimeout` on cancellation in JSDispatcher. - * Remove delayed task on cancellation from internal data structure on JVM. -* Introduce `ticker` function to create "ticker channels" (see #327): - * It provides analogue of RX `Observable.timer` for coroutine channels. - * It is currently supported on JVM only. -* Add a test-helper class `TestCoroutineContext` (see #297, PR by @streetsofboston). - * It is currently supported on JVM only. - * Ticker channels (#327) are not yet compatible with it. -* Implement a better way to set `CoroutineContext.DEBUG` value (see #316, PR by @dmytrodanylyk): - * Made `CoroutineContext.DEBUG_PROPERTY_NAME` constant public. - * Introduce public constants with `"on"`, `"off"`, `"auto"` values. -* Introduce system property to control `CommonPool` parallelism (see #343): - * `CommonPool.DEFAULT_PARALLELISM_PROPERTY_NAME` constant is introduced with a value of "kotlinx.coroutines.default.parallelism". -* Include package-list files into documentation site (see #290). -* Fix various typos in docs (PRs by @paolop and @ArtsiomCh). - -## Version 0.22.5 - -* JS: Fixed main file reference in [NPM package](https://www.npmjs.com/package/kotlinx-coroutines-core) -* Added context argument to `Channel.filterNot` (PR by @jcornaz). -* Implemented debug `toString` for channels (see #185). - -## Version 0.22.4 - -* JS: Publish to NPM (see #229). -* JS: Use node-style dispatcher on ReactNative (see #236). -* [jdk8 integration](integration/kotlinx-coroutines-jdk8/README.md) improvements: - * Added conversion from `CompletionStage` to `Deferred` (see #262, PR by @jcornaz). - * Use fast path in `CompletionStage.await` and make it cancellable. - -## Version 0.22.3 - -* Fixed `produce` builder to close the channel on completion instead of cancelling it, which lead to lost elements with buffered channels (see #256). -* Don't use `ForkJoinPool` if there is a `SecurityManager` present to work around JNLP problems (see #216, PR by @NikolayMetchev). -* JS: Check for undefined `window.addEventListener` when choosing default coroutine dispatcher (see #230, PR by @ScottPierce). -* Update 3rd party dependencies: - * [kotlinx-coroutines-rx1](reactive/kotlinx-coroutines-rx1) to RxJava version `1.3.6`. - * [kotlinx-coroutines-rx2](reactive/kotlinx-coroutines-rx2) to RxJava version `2.1.9`. - * [kotlinx-coroutines-guava](integration/kotlinx-coroutines-guava) to Guava version `24.0-jre`. - -## Version 0.22.2 - -* Android: Use @Keep annotation on AndroidExceptionPreHandler to fix the problem on Android with minification enabled (see #214). -* Reactive: Added `awaitFirstOrDefault` and `awaitFirstOrNull` extensions (see #224, PR by @konrad-kaminski). -* Core: Fixed `withTimeout` and `withTimeoutOrNull` that should not use equals on result (see #212, PR by @konrad-kaminski). -* Core: Fixed hanged receive from a closed subscription of BroadcastChannel (see #226). -* IO: fixed error propagation (see https://github.com/ktorio/ktor/issues/301). -* Include common sources into sources jar file to work around KT-20971. -* Fixed bugs in documentation due to MPP. - -## Version 0.22.1 - -* Migrated to Kotlin 1.2.21. -* Improved `actor` builder documentation (see #210) and fixed bugs in rendered documentation due to multiplatform. -* Fixed `runBlocking` to properly support specified dispatchers (see #209). -* Fixed data race in `Job` implementation (it was hanging at `LockFreeLinkedList.helpDelete` on certain stress tests). -* `AbstractCoroutine.onCancellation` is invoked before cancellation handler that is set via `invokeOnCompletion`. -* Ensure that `launch` handles uncaught exception before another coroutine that uses `join` on it resumes (see #208). - -## Version 0.22 - -* Migrated to Kotlin 1.2.20. -* Introduced stable public API for `AbstractCoroutine`: - * Implements `Job`, `Continuation`, and `CoroutineScope`. - * Has overridable `onStart`, `onCancellation`, `onCompleted` and `onCompletedExceptionally` functions. - * Reactive integration modules are now implemented using public API only. - * Notifies onXXX before all the installed handlers, so `launch` handles uncaught exceptions before "joining" coroutines wakeup (see #208). - -## Version 0.21.2 - -* Fixed `openSubscription` extension for reactive `Publisher`/`Observable`/`Flowable` when used with `select { ... }` and added an optional `request` parameter to specify how many elements are requested from publisher in advance on subscription (see #197). -* Simplified implementation of `Channel.flatMap` using `toChannel` function to work around Android 5.0 APK install SIGSEGV (see #205). - -## Version 0.21.1 - -* Improved performance of coroutine dispatching (`DispatchTask` instance is no longer allocated). -* Fixed `Job.cancel` and `CompletableDeferred.complete` to support cancelling/completing states and properly wait for their children to complete on join/await (see #199). -* Fixed a bug in binary heap implementation (used internally by `delay`) which could have resulted in wrong delay time in rare circumstances. -* Coroutines library for [Kotlin/JS](js/README.md): - * `Promise.asDeferred` immediately installs handlers to avoid "Unhandled promise rejection" warning. - * Use `window.postMessage` instead of `setTimeout` for coroutines inside the browser to avoid timeout throttling (see #194). - * Use custom queue in `Window.awaitAnimationFrame` to align all animations and reduce overhead. - * Introduced `Window.asCoroutineDispatcher()` extension function. - -## Version 0.21 - -* Migrated to Kotlin 1.2.10. -* Coroutines library for [Kotlin/JS](js/README.md) and [multiplatform projects](https://kotlinlang.org/docs/reference/multiplatform.html) (see #33): - * `launch` and `async` coroutine builders. - * `Job` and `Deferred` light-weight future with cancellation support. - * `delay` and `yield` top-level suspending functions. - * `await` extension for JS `Promise` and `asPromise`/`asDeferred` conversions. - * `promise` coroutine builder. - * `Job()` and `CompletableDeferred()` factories. - * Full support for parent-child coroutine hierarchies. - * `Window.awaitAnimationFrame` extension function. - * [Sample frontend Kotlin/JS application](js/example-frontend-js/README.md) with coroutine-driven animations. -* `run` is deprecated and renamed to `withContext` (see #134). -* `runBlocking` and `EventLoop` implementations optimized (see #190). - -## Version 0.20 - -* Migrated to Kotlin 1.2.0. -* Channels: - * Sequence-like `filter`, `map`, etc extensions on `ReceiveChannel` are introduced (see #88 by @fvasco and #69 by @konrad-kaminski). - * Introduced `ReceiveChannel.cancel` method. - * All operators on `ReceiveChannel` fully consume the original channel (`cancel` it when they are done) using a helper `consume` extension. - * Deprecated `ActorJob` and `ProducerJob`; `actor` now returns `SendChannel` and `produce` returns `ReceiveChannel` (see #127). - * `SendChannel.sendBlocking` extension method (see #157 by @@fvasco). -* Parent-child relations between coroutines: - * Introduced an optional `parent` job parameter for all coroutine builders so that code with an explict parent `Job` is more natural. - * Added `parent` parameter to `CompletableDeferred` constructor. - * Introduced `Job.children` property. - * `Job.cancelChildren` is now an extension (member is deprecated and hidden). - * `Job.joinChildren` extension is introduced. - * Deprecated `Job.attachChild` as a error-prone API. - * Fixed StackOverflow when waiting for a lot of completed children that did not remove their handlers from the parent. -* Use `java.util.ServiceLoader` to find default instances of `CoroutineExceptionHandler`. -* Android UI integration: - * Use `Thread.getUncaughtExceptionPreHandler` to make sure that exceptions are logged before crash (see #148). - * Introduce `UI.awaitFrame` for animation; added sample coroutine-based animation application for Android [here](ui/kotlinx-coroutines-android/animation-app). - * Fixed `delay(Long.MAX_VALUE)` (see #161) -* Added missing `DefaultDispatcher` on some reactive operators (see #174 by @fvasco) -* Fixed `actor` and `produce` so that a cancellation of a Job cancels the underlying channel (closes and removes all the pending messages). -* Fixed sporadic failure of `example-context-06` (see #160) -* Fixed hang of `Job.start` on lazy coroutine with attached `invokeOnCompletion` handler. -* A more gradual introduction to `runBlocking` and coroutines in the [guide](docs/topics/coroutines-guide.md) (see #166). - -## Version 0.19.3 - -* Fixed `send`/`openSubscription` race in `ArrayBroadcastChannel`. - This race lead to stalled (hanged) `send`/`receive` invocations. -* Project build has been migrated to Gradle. - -## Version 0.19.2 - -* Fixed `ArrayBroadcastChannel` receive of stale elements on `openSubscription`. - Only elements that are sent after invocation of `openSubscription` are received now. -* Added a default value for `context` parameter to `rxFlowable` (see #146 by @PhilGlass). -* Exception propagation logic from cancelled coroutines is adjusted (see #152): - * When cancelled coroutine crashes due to some other exception, this other exception becomes the cancellation reason - of the coroutine, while the original cancellation reason is suppressed. - * `UnexpectedCoroutineException` is no longer used to report those cases as is removed. - * This fixes a race between crash of CPU-consuming coroutine and cancellation which resulted in an unhandled exception - and lead to crashes on Android. -* `run` uses cancelling state & propagates exceptions when cancelled (see #147): - * When coroutine that was switched into a different dispatcher using `run` is cancelled, the run invocation does not - complete immediately, but waits until the body completes. - * If the body completes with exception, then this exception is propagated. -* No `Job` in `newSingleThreadContext` and `newFixedThreadPoolContext` anymore (see #149, #151): - * This resolves the common issue of using `run(ctx)` where ctx comes from either `newSingleThreadContext` or - `newFixedThreadPoolContext` invocation. They both used to return a combination of dispatcher + job, - and this job was overriding the parent job, thus preventing propagation of cancellation. Not anymore. - * `ThreadPoolDispatcher` class is now public and is the result type for both functions. - It has the `close` method to release the thread pool. - -## Version 0.19.1 - -* Failed parent Job cancels all children jobs, then waits for them them. - This makes parent-child hierarchies easier to get working right without - having to use `try/catch` or other exception handlers. -* Fixed a race in `ArrayBroadcastChannel` between `send` and `openChannel` invocations - (see #138). -* Fixed quite a rare race in `runBlocking` that resulted in `AssertionError`. - Unfortunately, cannot write a reliable stress-test to reproduce it. -* Updated Reactor support to leverage Bismuth release train - (contributed by @sdeleuze, see PR #141) - -## Version 0.19 - -* This release is published to Maven Central. -* `DefaultDispatcher` is introduced (see #136): - * `launch`, `async`, `produce`, `actor` and other integration-specific coroutine builders now use - `DefaultDispatcher` as the default value for their `context` parameter. - * When a context is explicitly specified, `newCoroutineContext` function checks if there is any - interceptor/dispatcher defined in the context and uses `DefaultDispatcher` if there is none. - * `DefaultDispatcher` is currently defined to be equal to `CommonPool`. - * Examples in the [guide](docs/topics/coroutines-guide.md) now start with `launch { ... }` code and explanation on the nature - and the need for coroutine context starts in "Coroutine context and dispatchers" section. -* Parent coroutines now wait for their children (see #125): - * Job _completing_ state is introduced in documentation as a state in which parent coroutine waits for its children. - * `Job.attachChild` and `Job.cancelChildren` are introduced. - * `Job.join` now always checks cancellation status of invoker coroutine for predictable behavior when joining - failed child coroutine. - * `Job.cancelAndJoin` extension is introduced. - * `CoroutineContext.cancel` and `CoroutineContext.cancelChildren` extensions are introduced for convenience. - * `withTimeout`/`withTimeoutOrNull` blocks become proper coroutines that have `CoroutineScope` and wait for children, too. - * Diagnostics in cancellation and unexpected exception messages are improved, - coroutine name is included in debug mode. - * Fixed cancellable suspending functions to throw `CancellationException` (as was documented before) even when - the coroutine is cancelled with another application-specific exception. - * `JobCancellationException` is introduced as a specific subclass of `CancellationException` which is - used for coroutines that are cancelled without cause and to wrap application-specific exceptions. - * `Job.getCompletionException` is renamed to `Job.getCancellationException` and return a wrapper exception if needed. - * Introduced `Deferred.getCompletionExceptionOrNull` to get not-wrapped exception result of `async` task. - * Updated docs for `Job` & `Deferred` to explain parent/child relations. -* `select` expression is modularized: - * `SelectClause(0,1,2)` interfaces are introduced, so that synchronization - constructs can define their select clauses without having to modify - the source of the `SelectBuilder` in `kotlinx-corounes-core` module. - * `Job.onJoin`, `Deferred.onAwait`, `Mutex.onLock`, `SendChannel.onSend`, `ReceiveChannel.onReceive`, etc - that were functions before are now properties returning the corresponding select clauses. Old functions - are left in bytecode for backwards compatibility on use-site, but any outside code that was implementing those - interfaces by itself must be updated. - * This opens road to moving channels into a separate module in future updates. -* Renamed `TimeoutException` to `TimeoutCancellationException` (old name is deprecated). -* Fixed various minor problems: - * JavaFx toolkit is now initialized by `JavaFx` context (see #108). - * Fixed lost ACC_STATIC on <clinit> methods (see #116). - * Fixed link to source code from documentation (see #129). - * Fixed `delay` in arbitrary contexts (see #133). -* `kotlinx-coroutines-io` module is introduced. It is a work-in-progress on `ByteReadChannel` and `ByteWriteChannel` - interfaces, their implementations, and related classes to enable convenient coroutine integration with various - asynchronous I/O libraries and sockets. It is currently _unstable_ and **will change** in the next release. - -## Version 0.18 - -* Kotlin 1.1.4 is required to use this version, which enables: - * `withLock` and `consumeEach` functions are now inline suspend functions. - * `JobSupport` class implementation is optimized (one fewer field). -* `TimeoutException` is public (see #89). -* Improvements to `Mutex` (courtesy of @fvasco): - * Introduced `holdsLock` (see #92). - * Improved documentation on `Mutex` fairness (see #90). -* Fixed NPE when `ArrayBroadcastChannel` is closed concurrently with receive (see #97). -* Fixed bug in internal class LockFreeLinkedList that resulted in ISE under stress in extremely rare circumstances. -* Integrations: - * [quasar](integration/kotlinx-coroutines-quasar): Introduced integration with suspendable JVM functions - that are instrumented with [Parallel Universe Quasar](https://docs.paralleluniverse.co/quasar/) - (thanks to the help of @pron). - * [reactor](reactive/kotlinx-coroutines-reactor): Replaced deprecated `setCancellation` with `onDipose` and - updated to Aluminium-SR3 release (courtesy of @yxf07, see #96) - * [jdk8](integration/kotlinx-coroutines-jdk8): Added adapters for `java.time` classes (courtesy of @fvasco, see #93) - -## Version 0.17 - -* `CompletableDeferred` is introduced as a set-once event-like communication primitive (see #70). - * [Coroutines guide](docs/topics/coroutines-guide.md) uses it in a section on actors. - * `CompletableDeferred` is an interface with private impl (courtesy of @fvasco, see #86). - * It extends `Deferred` interface with `complete` and `completeExceptionally` functions. -* `Job.join` and `Deferred.await` wait until a cancelled coroutine stops execution (see #64). - * `Job` and `Deferred` have a new _cancelling_ state which they enter on invocation of `cancel`. - * `Job.invokeOnCompletion` has an additional overload with `onCancelling: Boolean` parameter to - install handlers that are fired as soon as coroutine enters _cancelling_ state as opposed - to waiting until it _completes_. - * Internal `select` implementation is refactored to decouple it from `JobSupport` internal class - and to optimize its state-machine. - * Internal `AbstractCoroutine` class is refactored so that it is extended only by true coroutines, - all of which support the new _cancelling_ state. -* `CoroutineScope.context` is renamed to `coroutineContext` to avoid conflicts with other usages of `context` - in applications (like Android context, see #75). -* `BroadcastChannel.open` is renamed to `openSubscription` (see #54). -* Fixed `StackOverflowError` in a convoy of `Mutex.unlock` invokers with `Unconfined` dispatcher (see #80). -* Fixed `SecurityException` when trying to use coroutines library with installed `SecurityManager`. -* Fixed a bug in `withTimeoutOrNull` in case with nested timeouts when coroutine was cancelled before it was - ever suspended. -* Fixed a minor problem with `awaitFirst` on reactive streams that would have resulted in spurious stack-traces printed - on the console when used with publishers/observables that continue to invoke `onNext` despite being cancelled/disposed - (which they are technically allowed to do by specification). -* All factory functions for various interfaces are implemented as top-level functions - (affects `Job`, `Channel`, `BroadcastChannel`, `Mutex`, `EventLoop`, and `CoroutineExceptionHandler`). - Previous approach of using `operator invoke` on their companion objects is deprecated. -* Nicer-to-use debug `toString` implementations for coroutine dispatcher tasks and continuations. -* A default dispatcher for `delay` is rewritten and now shares code with `EventLoopImpl` that is used by - `runBlocking`. It internally supports non-default `TimeSource` so that delay-using tests can be written - with "virtual time" by replacing their time source for the duration of tests (this feature is not available - outside of the library). - -## Version 0.16 - -* Coroutines that are scheduled for execution are cancellable by default now - * `suspendAtomicCancellableCoroutine` function is introduced for funs like - `send`/`receive`/`receiveOrNull` that require atomic cancellation - (they cannot be cancelled after decision was made) - * Coroutines started with default mode using - `async`/`launch`/`actor` builders can be cancelled before their execution starts - * `CoroutineStart.ATOMIC` is introduced as a start mode to specify that - coroutine cannot be cancelled before its execution starts - * `run` function is also cancellable in the same way and accepts an optional - `CoroutineStart` parameter to change this default. -* `BroadcastChannel` factory function is introduced -* `CoroutineExceptionHandler` factory function is introduced by @konrad-kaminski -* [`integration`](integration) directory is introduced for all 3rd party integration projects - * It has [contribution guidelines](integration/README.md#contributing) and contributions from community are welcome - * Support for Guava `ListenableFuture` in the new [`kotlinx-coroutines-guava`](integration/kotlinx-coroutines-guava) module - * Rx1 Scheduler support by @konrad-kaminski -* Fixed a number of `Channel` and `BroadcastChannel` implementation bugs related to concurrent - send/close/close of channels that lead to hanging send, offer or close operations (see #66). - Thanks to @chrisly42 and @cy6erGn0m for finding them. -* Fixed `withTimeoutOrNull` which was returning `null` on timeout of inner or outer `withTimeout` blocks (see #67). - Thanks to @gregschlom for finding the problem. -* Fixed a bug where `Job` fails to dispose a handler when it is the only handler by @uchuhimo - -## Version 0.15 - -* Switched to Kotlin version 1.1.2 (can still be used with 1.1.0). -* `CoroutineStart` enum is introduced for `launch`/`async`/`actor` builders: - * The usage of `luanch(context, start = false)` is deprecated and is replaced with - `launch(context, CoroutineStart.LAZY)` - * `CoroutineStart.UNDISPATCHED` is introduced to start coroutine execution immediately in the invoker thread, - so that `async(context, CoroutineStart.UNDISPATCHED)` is similar to the behavior of C# `async`. - * [Guide to UI programming with coroutines](ui/coroutines-guide-ui.md) mentions the use of it to optimize - the start of coroutines from UI threads. -* Introduced `BroadcastChannel` interface in `kotlinx-coroutines-core` module: - * It extends `SendChannel` interface and provides `open` function to create subscriptions. - * Subscriptions are represented with `SubscriptionReceiveChannel` interface. - * The corresponding `SubscriptionReceiveChannel` interfaces are removed from [reactive](reactive) implementation - modules. They use an interface defined in `kotlinx-coroutines-core` module. - * `ConflatedBroadcastChannel` implementation is provided for state-observation-like use-cases, where a coroutine or a - regular code (in UI, for example) updates the state that subscriber coroutines shall react to. - * `ArrayBroadcastChannel` implementation is provided for event-bus-like use-cases, where a sequence of events shall - be received by multiple subscribers without any omissions. - * [Guide to reactive streams with coroutines](reactive/coroutines-guide-reactive.md) includes - "Rx Subject vs BroadcastChannel" section. -* Pull requests from Konrad Kamiński are merged into reactive stream implementations: - * Support for Project Reactor `Mono` and `Flux`. - See [`kotlinx-coroutines-reactor`](reactive/kotlinx-coroutines-reactor) module. - * Implemented Rx1 `Completable.awaitCompleted`. - * Added support for Rx2 `Maybe`. -* Better timeout support: - * Introduced `withTimeoutOrNull` function. - * Implemented `onTimeout` clause for `select` expressions. - * Fixed spurious concurrency inside `withTimeout` blocks on their cancellation. - * Changed behavior of `withTimeout` when `CancellationException` is suppressed inside the block. - Invocation of `withTimeout` now always returns the result of execution of its inner block. -* The `channel` property in `ActorScope` is promoted to a wider `Channel` type, so that an actor - can have an easy access to its own inbox send channel. -* Renamed `Mutex.withMutex` to `Mutex.withLock`, old name is deprecated. - -## Version 0.14 - -* Switched to Kotlin version 1.1.1 (can still be used with 1.1.0). -* Introduced `consumeEach` helper function for channels and reactive streams, Rx 1.x, and Rx 2.x. - * It ensures that streams are unsubscribed from on any exception. - * Iteration with `for` loop on reactive streams is **deprecated**. - * [Guide to reactive streams with coroutines](reactive/coroutines-guide-reactive.md) is updated virtually - all over the place to reflect these important changes. -* Implemented `awaitFirstOrDefault` extension for reactive streams, Rx 1.x, and Rx 2.x. -* Added `Mutex.withMutex` helper function. -* `kotlinx-coroutines-android` module has `provided` dependency on of Android APIs to - eliminate warnings when using it in android project. - -## Version 0.13 - -* New `kotlinx-coroutinex-android` module with Android `UI` context implementation. -* Introduced `whileSelect` convenience function. -* Implemented `ConflatedChannel`. -* Renamed various `toXXX` conversion functions to `asXXX` (old names are deprecated). -* `run` is optimized with fast-path case and no longer has `CoroutineScope` in its block. -* Fixed dispatching logic of `withTimeout` (removed extra dispatch). -* `EventLoop` that is used by `runBlocking` now implements Delay, giving more predictable test behavior. -* Various refactorings related to resource management and timeouts: - * `Job.Registration` is renamed to `DisposableHandle`. - * `EmptyRegistration` is renamed to `NonDisposableHandle`. - * `Job.unregisterOnCompletion` is renamed to `Job.disposeOnCompletion`. - * `Delay.invokeOnTimeout` is introduced. - * `withTimeout` now uses `Delay.invokeOnTimeout` when available. -* A number of improvement for reactive streams and Rx: - * Introduced `rxFlowable` builder for Rx 2.x. - * `Scheduler.asCoroutineDispatcher` extension for Rx 2.x. - * Fixed bug with sometimes missing `onComplete` in `publish`, `rxObservable`, and `rxFlowable` builders. - * Channels that are open for reactive streams are now `Closeable`. - * Fixed `CompletableSource.await` and added test for it. - * Removed `rx.Completable.await` due to name conflict. -* New documentation: - * [Guide to UI programming with coroutines](ui/coroutines-guide-ui.md) - * [Guide to reactive streams with coroutines](reactive/coroutines-guide-reactive.md) -* Code is published to JCenter repository. - -## Version 0.12 - -* Switched to Kotlin version 1.1.0 release. -* Reworked and updated utilities for - [Reactive Streams](kotlinx-coroutines-reactive), - [Rx 1.x](kotlinx-coroutines-rx1), and - [Rx 2.x](kotlinx-coroutines-rx2) with library-specific - coroutine builders, suspending functions, converters and iteration support. -* `LinkedListChannel` with unlimited buffer (`offer` always succeeds). -* `onLock` select clause and an optional `owner` parameter in all `Mutex` functions. -* `selectUnbiased` function. -* `actor` coroutine builder. -* Couple more examples for "Shared mutable state and concurrency" section and - "Channels are fair" section with ping-pong table example - in [coroutines guide](docs/topics/coroutines-guide.md). - -## Version 0.11-rc - -* `select` expression with onJoin/onAwait/onSend/onReceive clauses. -* `Mutex` is moved to `kotlinx.coroutines.sync` package. -* `ClosedSendChannelException` is a subclass of `CancellationException` now. -* New sections on "Shared mutable state and concurrency" and "Select expression" - in [coroutines guide](docs/topics/coroutines-guide.md). - -## Version 0.10-rc - -* Switched to Kotlin version 1.1.0-rc-91. -* `Mutex` synchronization primitive is introduced. -* `buildChannel` is renamed to `produce`, old name is deprecated. -* `Job.onCompletion` is renamed to `Job.invokeOnCompletion`, old name is deprecated. -* `delay` implementation in Swing, JavaFx, and scheduled executors is fixed to avoid an extra dispatch. -* `CancellableContinuation.resumeUndispatched` is introduced to make this efficient implementation possible. -* Remove unnecessary creation of `CancellationException` to improve performance, plus other performance improvements. -* Suppress deprecated and internal APIs from docs. -* Better docs at top level with categorized summary of classes and functions. - -## Version 0.8-beta - -* `defer` coroutine builder is renamed to `async`. -* `lazyDefer` is deprecated, `async` has an optional `start` parameter instead. -* `LazyDeferred` interface is deprecated, lazy start functionality is integrated into `Job` interface. -* `launch` has an optional `start` parameter for lazily started coroutines. -* `Job.start` and `Job.isCompleted` are introduced. -* `Deferred.isCompletedExceptionally` and `Deferred.isCancelled` are introduced. -* `Job.getInactiveCancellationException` is renamed to `getCompletionException`. -* `Job.join` is now a member function. -* Internal `JobSupport` state machine is enhanced to support _new_ (not-started-yet) state. - So, lazy coroutines do not need a separate state variable to track their started/not-started (new/active) status. -* Exception transparency in `Job.cancel` (original cause is rethrown). -* Clarified possible states for `Job`/`CancellableContinuation`/`Deferred` in docs. -* Example on async-style functions and links to API reference site from [coroutines guide](docs/topics/coroutines-guide.md). - -## Version 0.7-beta - -* Buffered and unbuffered channels are introduced: `Channel`, `SendChannel`, and `ReceiveChannel` interfaces, - `RendezvousChannel` and `ArrayChannel` implementations, `Channel()` factory function and `buildChannel{}` - coroutines builder. -* `Here` context is renamed to `Unconfined` (the old name is deprecated). -* A [guide on coroutines](docs/topics/coroutines-guide.md) is expanded: sections on contexts and channels. - -## Version 0.6-beta - -* Switched to Kotlin version 1.1.0-beta-37. -* A [guide on coroutines](docs/topics/coroutines-guide.md) is expanded. - -## Version 0.5-beta - -* Switched to Kotlin version 1.1.0-beta-22 (republished version). -* Removed `currentCoroutineContext` and related thread-locals without replacement. - Explicitly pass coroutine context around if needed. -* `lazyDefer(context) {...}` coroutine builder and `LazyDeferred` interface are introduced. -* The default behaviour of all coroutine dispatchers is changed to always schedule execution of new coroutine - for later in this thread or thread pool. Correspondingly, `CoroutineDispatcher.isDispatchNeeded` function - has a default implementation that returns `true`. -* `NonCancellable` context is introduced. -* Performance optimizations for cancellable continuations (fewer objects created). -* A [guide on coroutines](docs/topics/coroutines-guide.md) is added. - -## Version 0.4-beta - -* Switched to Kotlin version 1.1.0-beta-18 (republished version). -* `CoroutineDispatcher` methods now have `context` parameter. -* Introduced `CancellableContinuation.isCancelled` -* Introduced `EventLoop` dispatcher and made it a default for `runBlocking { ... }` -* Introduced `CoroutineScope` interface with `isActive` and `context` properties; - standard coroutine builders include it as receiver for convenience. -* Introduced `Executor.toCoroutineDispatcher()` extension. -* Delay scheduler thread is not daemon anymore, but times out automatically. -* Debugging facilities in `newCoroutineContext` can be explicitly disabled with `-Dkotlinx.coroutines.debug=off`. -* xxx-test files are renamed to xxx-example for clarity. -* Fixed NPE in Job implementation when starting coroutine with already cancelled parent job. -* Support cancellation in `kotlinx-coroutines-nio` module +* Kotlin version is updated to 1.8.20 +* Atomicfu version is updated to 0.20.2. +* `JavaFx` version is updated to 17.0.2 in `kotlinx-coroutines-javafx` (#3671).. +* JPMS is supported (#2237). Thanks @lion7! +* `BroadcastChannel` and all the corresponding API are deprecated (#2680). +* Added all supported K/N targets (#3601, #812, #855). +* K/N `Dispatchers.Default` is backed by the number of threads equal to the number of available cores (#3366). +* Fixed an issue where some coroutines' internal exceptions were not properly serializable (#3328). +* Introduced `Job.parent` API (#3201). +* Fixed a bug when `TestScheduler` leaked cancelled jobs (#3398). +* `TestScope.timeSource` now provides comparable time marks (#3617). Thanks @hfhbd! +* Fixed an issue when cancelled `withTimeout` handles were preserved in JS runtime (#3440). +* Ensure `awaitFrame` only awaits a single frame when used from the main looper (#3432). Thanks @pablobaxter! +* Obsolete `Class-Path` attribute was removed from `kotlinx-coroutines-debug.jar` manifest (#3361). +* Fixed a bug when `updateThreadContext` operated on the parent context (#3411). +* Added new `Flow.filterIsInstance` extension (#3240). +* `Dispatchers.Default` thread name prefixes are now configurable with system property (#3231). +* Added `Flow.timeout` operator as `@FlowPreview` (#2624). Thanks @pablobaxter! +* Improved the performance of the `future` builder in case of exceptions (#3475). Thanks @He-Pin! +* `Mono.awaitSingleOrNull` now waits for the `onComplete` signal (#3487). +* `Channel.isClosedForSend` and `Channel.isClosedForReceive` are promoted from experimental to delicate (#3448). +* Fixed a data race in native `EventLoop` (#3547). +* `Dispatchers.IO.limitedParallelism(valueLargerThanIOSize)` no longer creates an additional wrapper (#3442). Thanks @dovchinnikov! +* Various `@FlowPreview` and `@ExperimentalCoroutinesApi` are promoted to experimental and stable respectively (#3542, #3097, #3548). +* Performance improvements in `Dispatchers.Default` and `Dispatchers.IO` (#3416, #3418). +* Fixed a bug when internal `suspendCancellableCoroutineReusable` might have hanged (#3613). +* Introduced internal API to process events in the current system dispatcher (#3439). +* Global `CoroutineExceptionHandler` is no longer invoked in case of unprocessed `future` failure (#3452). +* Performance improvements and reduced thread-local pressure for the `withContext` operator (#3592). +* Improved performance of `DebugProbes` (#3527). +* Fixed a bug when the coroutine debugger might have detected the state of a coroutine incorrectly (#3193). +* `CoroutineDispatcher.asExecutor()` runs tasks without dispatching if the dispatcher is unconfined (#3683). Thanks @odedniv! +* `SharedFlow.toMutableList` and `SharedFlow.toSet` lints are introduced (#3706). +* `Channel.invokeOnClose` is promoted to stable API (#3358). +* Improved lock contention in `Dispatchers.Default` and `Dispatchers.IO` during the startup phase (#3652). +* Fixed a bug that led to threads oversubscription in `Dispatchers.Default` (#3642). +* Fixed a bug that allowed `limitedParallelism` to perform dispatches even after the underlying dispatcher was closed (#3672). +* Fixed a bug that prevented stacktrace recovery when the exception's constructor from `cause` was selected (#3714). +* Improved sanitizing of stracktrace-recovered traces (#3714). +* Introduced an internal flag to disable uncaught exceptions reporting in tests as a temporary migration mechanism (#3736). +* Various documentation improvements and fixes. + +Changelog for previous versions may be found in [CHANGES_UP_TO_1.7.md](CHANGES_UP_TO_1.7.md) diff --git a/CHANGES_UP_TO_1.7.md b/CHANGES_UP_TO_1.7.md new file mode 100644 index 00000000..c59e3b30 --- /dev/null +++ b/CHANGES_UP_TO_1.7.md @@ -0,0 +1,1606 @@ +# Change log for kotlinx.coroutines + +## Version 1.7.0 + +### Core API significant improvements + +* New `Channel` implementation with significant performance improvements across the API (#3621). +* New `select` operator implementation: faster, more lightweight, and more robust (#3020). +* `Mutex` and `Semaphore` now share the same underlying data structure (#3020). +* `Dispatchers.IO` is added to K/N (#3205) + * `newFixedThreadPool` and `Dispatchers.Default` implementations on K/N were wholly rewritten to support graceful growth under load (#3595). +* `kotlinx-coroutines-test` rework: + - Add the `timeout` parameter to `runTest` for the whole-test timeout, 10 seconds by default (#3270). This replaces the configuration of quiescence timeouts, which is now deprecated (#3603). + - The `withTimeout` exception messages indicate if the timeout used the virtual time (#3588). + - `TestCoroutineScheduler`, `runTest`, and `TestScope` API are promoted to stable (#3622). + - `runTest` now also fails if there were uncaught exceptions in coroutines not inherited from the test coroutine (#1205). + +### Breaking changes + +* Old K/N memory model is no longer supported (#3375). +* New generic upper bounds were added to reactive integration API where the language since 1.8.0 dictates (#3393). +* `kotlinx-coroutines-core` and `kotlinx-coroutines-jdk8` artifacts were merged into a single artifact (#3268). +* Artificial stackframes in stacktrace recovery no longer contain the `\b` symbol and are now navigable in IDE and supplied with proper documentation (#2291). +* `CoroutineContext.isActive` returns `true` for contexts without any job in them (#3300). + +### Bug fixes and improvements + +* Kotlin version is updated to 1.8.20 +* Atomicfu version is updated to 0.20.2. +* `JavaFx` version is updated to 17.0.2 in `kotlinx-coroutines-javafx` (#3671).. +* JPMS is supported (#2237). Thanks @lion7! +* `BroadcastChannel` and all the corresponding API are deprecated (#2680). +* Added all supported K/N targets (#3601, #812, #855). +* K/N `Dispatchers.Default` is backed by the number of threads equal to the number of available cores (#3366). +* Fixed an issue where some coroutines' internal exceptions were not properly serializable (#3328). +* Introduced `Job.parent` API (#3201). +* Fixed a bug when `TestScheduler` leaked cancelled jobs (#3398). +* `TestScope.timeSource` now provides comparable time marks (#3617). Thanks @hfhbd! +* Fixed an issue when cancelled `withTimeout` handles were preserved in JS runtime (#3440). +* Ensure `awaitFrame` only awaits a single frame when used from the main looper (#3432). Thanks @pablobaxter! +* Obsolete `Class-Path` attribute was removed from `kotlinx-coroutines-debug.jar` manifest (#3361). +* Fixed a bug when `updateThreadContext` operated on the parent context (#3411). +* Added new `Flow.filterIsInstance` extension (#3240). +* `Dispatchers.Default` thread name prefixes are now configurable with system property (#3231). +* Added `Flow.timeout` operator as `@FlowPreview` (#2624). Thanks @pablobaxter! +* Improved the performance of the `future` builder in case of exceptions (#3475). Thanks @He-Pin! +* `Mono.awaitSingleOrNull` now waits for the `onComplete` signal (#3487). +* `Channel.isClosedForSend` and `Channel.isClosedForReceive` are promoted from experimental to delicate (#3448). +* Fixed a data race in native `EventLoop` (#3547). +* `Dispatchers.IO.limitedParallelism(valueLargerThanIOSize)` no longer creates an additional wrapper (#3442). Thanks @dovchinnikov! +* Various `@FlowPreview` and `@ExperimentalCoroutinesApi` are promoted to experimental and stable respectively (#3542, #3097, #3548). +* Performance improvements in `Dispatchers.Default` and `Dispatchers.IO` (#3416, #3418). +* Fixed a bug when internal `suspendCancellableCoroutineReusable` might have hanged (#3613). +* Introduced internal API to process events in the current system dispatcher (#3439). +* Global `CoroutineExceptionHandler` is no longer invoked in case of unprocessed `future` failure (#3452). +* Performance improvements and reduced thread-local pressure for the `withContext` operator (#3592). +* Improved performance of `DebugProbes` (#3527). +* Fixed a bug when the coroutine debugger might have detected the state of a coroutine incorrectly (#3193). +* `CoroutineDispatcher.asExecutor()` runs tasks without dispatching if the dispatcher is unconfined (#3683). Thanks @odedniv! +* `SharedFlow.toMutableList` and `SharedFlow.toSet` lints are introduced (#3706). +* `Channel.invokeOnClose` is promoted to stable API (#3358). +* Improved lock contention in `Dispatchers.Default` and `Dispatchers.IO` during the startup phase (#3652). +* Fixed a bug that led to threads oversubscription in `Dispatchers.Default` (#3642). +* Fixed a bug that allowed `limitedParallelism` to perform dispatches even after the underlying dispatcher was closed (#3672). +* Fixed a bug that prevented stacktrace recovery when the exception's constructor from `cause` was selected (#3714). +* Improved sanitizing of stracktrace-recovered traces (#3714). +* Introduced an internal flag to disable uncaught exceptions reporting in tests as a temporary migration mechanism (#3736). +* Various documentation improvements and fixes. + +### Changelog relative to version 1.7.0-RC + +* Fixed a bug that prevented stacktrace recovery when the exception's constructor from `cause` was selected (#3714). +* Improved sanitizing of stracktrace-recovered traces (#3714). +* Introduced an internal flag to disable uncaught exceptions reporting in tests as a temporary migration mechanism (#3736). + +## Version 1.7.0-RC + +* Kotlin version is updated to 1.8.20. +* Atomicfu version is updated to 0.20.2. +* `JavaFx` version is updated to 17.0.2 in `kotlinx-coroutines-javafx` (#3671). +* `previous-compilation-data.bin` file is removed from JAR resources (#3668). +* `CoroutineDispatcher.asExecutor()` runs tasks without dispatching if the dispatcher is unconfined (#3683). Thanks @odedniv! +* `SharedFlow.toMutableList` lint overload is undeprecated (#3706). +* `Channel.invokeOnClose` is promoted to stable API (#3358). +* Improved lock contention in `Dispatchers.Default` and `Dispatchers.IO` during the startup phase (#3652). +* Fixed a bug that led to threads oversubscription in `Dispatchers.Default` (#3642). +* Fixed a bug that allowed `limitedParallelism` to perform dispatches even after the underlying dispatcher was closed (#3672). +* Restored binary compatibility of previously experimental `TestScope.runTest(Long)` (#3673). + +## Version 1.7.0-Beta + +### Core API significant improvements + +* New `Channel` implementation with significant performance improvements across the API (#3621). +* New `select` operator implementation: faster, more lightweight, and more robust (#3020). +* `Mutex` and `Semaphore` now share the same underlying data structure (#3020). +* `Dispatchers.IO` is added to K/N (#3205) + * `newFixedThreadPool` and `Dispatchers.Default` implementations on K/N were wholly rewritten to support graceful growth under load (#3595). +* `kotlinx-coroutines-test` rework: + - Add the `timeout` parameter to `runTest` for the whole-test timeout, 10 seconds by default (#3270). This replaces the configuration of quiescence timeouts, which is now deprecated (#3603). + - The `withTimeout` exception messages indicate if the timeout used the virtual time (#3588). + - `TestCoroutineScheduler`, `runTest`, and `TestScope` API are promoted to stable (#3622). + - `runTest` now also fails if there were uncaught exceptions in coroutines not inherited from the test coroutine (#1205). + +### Breaking changes + +* Old K/N memory model is no longer supported (#3375). +* New generic upper bounds were added to reactive integration API where the language since 1.8.0 dictates (#3393). +* `kotlinx-coroutines-core` and `kotlinx-coroutines-jdk8` artifacts were merged into a single artifact (#3268). +* Artificial stackframes in stacktrace recovery no longer contain the `\b` symbol and are now navigable in IDE and supplied with proper documentation (#2291). +* `CoroutineContext.isActive` returns `true` for contexts without any job in them (#3300). + +### Bug fixes and improvements + +* Kotlin version is updated to 1.8.10. +* JPMS is supported (#2237). Thanks @lion7! +* `BroadcastChannel` and all the corresponding API are deprecated (#2680). +* Added all supported K/N targets (#3601, #812, #855). +* K/N `Dispatchers.Default` is backed by the number of threads equal to the number of available cores (#3366). +* Fixed an issue where some coroutines' internal exceptions were not properly serializable (#3328). +* Introduced `Job.parent` API (#3201). +* Fixed a bug when `TestScheduler` leaked cancelled jobs (#3398). +* `TestScope.timeSource` now provides comparable time marks (#3617). Thanks @hfhbd! +* Fixed an issue when cancelled `withTimeout` handles were preserved in JS runtime (#3440). +* Ensure `awaitFrame` only awaits a single frame when used from the main looper (#3432). Thanks @pablobaxter! +* Obsolete `Class-Path` attribute was removed from `kotlinx-coroutines-debug.jar` manifest (#3361). +* Fixed a bug when `updateThreadContext` operated on the parent context (#3411). +* Added new `Flow.filterIsInstance` extension (#3240). +* `Dispatchers.Default` thread name prefixes are now configurable with system property (#3231). +* Added `Flow.timeout` operator as `@FlowPreview` (#2624). Thanks @pablobaxter! +* Improved the performance of the `future` builder in case of exceptions (#3475). Thanks @He-Pin! +* `Mono.awaitSingleOrNull` now waits for the `onComplete` signal (#3487). +* `Channel.isClosedForSend` and `Channel.isClosedForReceive` are promoted from experimental to delicate (#3448). +* Fixed a data race in native `EventLoop` (#3547). +* `Dispatchers.IO.limitedParallelism(valueLargerThanIOSize)` no longer creates an additional wrapper (#3442). Thanks @dovchinnikov! +* Various `@FlowPreview` and `@ExperimentalCoroutinesApi` are promoted to experimental and stable respectively (#3542, #3097, #3548). +* Performance improvements in `Dispatchers.Default` and `Dispatchers.IO` (#3416, #3418). +* Fixed a bug when internal `suspendCancellableCoroutineReusable` might have hanged (#3613). +* Introduced internal API to process events in the current system dispatcher (#3439). +* Global `CoroutineExceptionHandler` is no longer invoked in case of unprocessed `future` failure (#3452). +* Performance improvements and reduced thread-local pressure for the `withContext` operator (#3592). +* Improved performance of `DebugProbes` (#3527). +* Fixed a bug when the coroutine debugger might have detected the state of a coroutine incorrectly (#3193). +* Various documentation improvements and fixes. + +## Version 1.6.4 + +* Added `TestScope.backgroundScope` for launching coroutines that perform work in the background and need to be cancelled at the end of the test (#3287). +* Fixed the POM of `kotlinx-coroutines-debug` having an incorrect reference to `kotlinx-coroutines-bom`, which cause the builds of Maven projects using the debug module to break (#3334). +* Fixed the `Publisher.await` functions in `kotlinx-coroutines-reactive` not ensuring that the `Subscriber` methods are invoked serially (#3360). Thank you, @EgorKulbachka! +* Fixed a memory leak in `withTimeout` on K/N with the new memory model (#3351). +* Added the guarantee that all `Throwable` implementations in the core library are serializable (#3328). +* Moved the documentation to <https://kotlinlang.org/api/kotlinx.coroutines/> (#3342). +* Various documentation improvements. + +## Version 1.6.3 + +* Updated atomicfu version to 0.17.3 (#3321), fixing the projects using this library with JS IR failing to build (#3305). + +## Version 1.6.2 + +* Fixed a bug with `ThreadLocalElement` not being correctly updated when the most outer `suspend` function was called directly without `kotlinx.coroutines` (#2930). +* Fixed multiple data races: one that might have been affecting `runBlocking` event loop, and a benign data race in `Mutex` (#3250, #3251). +* Obsolete `TestCoroutineContext` is removed, which fixes the `kotlinx-coroutines-test` JPMS package being split between `kotlinx-coroutines-core` and `kotlinx-coroutines-test` (#3218). +* Updated the ProGuard rules to further shrink the size of the resulting DEX file with coroutines (#3111, #3263). Thanks, @agrieve! +* Atomicfu is updated to `0.17.2`, which includes a more efficient and robust JS IR transformer (#3255). +* Kotlin is updated to `1.6.21`, Gradle version is updated to `7.4.2` (#3281). Thanks, @wojtek-kalicinski! +* Various documentation improvements. + +## Version 1.6.1 + +* Rollback of time-related functions dispatching on `Dispatchers.Main`. + This behavior was introduced in 1.6.0 and then found inconvenient and erroneous (#3106, #3113). +* Reworked the newly-introduced `CopyableThreadContextElement` to solve issues uncovered after the initial release (#3227). +* Fixed a bug with `ThreadLocalElement` not being properly updated in racy scenarios (#2930). +* Reverted eager loading of default `CoroutineExceptionHandler` that triggered ANR on some devices (#3180). +* New API to convert a `CoroutineDispatcher` to a Rx scheduler (#968, #548). Thanks @recheej! +* Fixed a memory leak with the very last element emitted from `flow` builder being retained in memory (#3197). +* Fixed a bug with `limitedParallelism` on K/N with new memory model throwing `ClassCastException` (#3223). +* `CoroutineContext` is added to the exception printed to the default `CoroutineExceptionHandler` to improve debuggability (#3153). +* Static memory consumption of `Dispatchers.Default` was significantly reduced (#3137). +* Updated slf4j version in `kotlinx-coroutines-slf4j` from 1.7.25 to 1.7.32. + +## Version 1.6.0 + +Note that this is a full changelog relative to the 1.5.2 version. Changelog relative to 1.6.0-RC3 can be found at the end. + +### kotlinx-coroutines-test rework + +* `kotlinx-coroutines-test` became a multiplatform library usable from K/JVM, K/JS, and K/N. +* Its API was completely reworked to address long-standing issues with consistency, structured concurrency and correctness (#1203, #1609, #2379, #1749, #1204, #1390, #1222, #1395, #1881, #1910, #1772, #1626, #1742, #2082, #2102, #2405, #2462 + ). +* The old API is deprecated for removal, but the new API is based on the similar concepts ([README](kotlinx-coroutines-test/README.md)), and the migration path is designed to be graceful: [migration guide](kotlinx-coroutines-test/MIGRATION.md). + +### Dispatchers + +* Introduced `CoroutineDispatcher.limitedParallelism` that allows obtaining a view of the original dispatcher with limited parallelism (#2919). +* `Dispatchers.IO.limitedParallelism` usages ignore the bound on the parallelism level of `Dispatchers.IO` itself to avoid starvation (#2943). +* Introduced new `Dispatchers.shutdown` method for containerized environments (#2558). +* `newSingleThreadContext` and `newFixedThreadPoolContext` are promoted to delicate API (#2919). + +### Breaking changes + +* When racing with cancellation, the `future` builder no longer reports unhandled exceptions into the global `CoroutineExceptionHandler`. Thanks @vadimsemenov! (#2774, #2791). +* `Mutex.onLock` is deprecated for removal (#2794). +* `Dispatchers.Main` is now used as the default source of time for `delay` and `withTimeout` when present(#2972). + * To opt-out from this behaviour, `kotlinx.coroutines.main.delay` system property can be set to `false`. +* Java target of coroutines build is now 8 instead of 6 (#1589). +* **Source-breaking change**: extension `collect` no longer resolves when used with a non-in-place argument of a functional type. This is a candidate for a fix, uncovered after 1.6.0, see #3107 for the additional details. + +### Bug fixes and improvements + +* Kotlin is updated to 1.6.0. +* Kotlin/Native [new memory model](https://blog.jetbrains.com/kotlin/2021/08/try-the-new-kotlin-native-memory-manager-development-preview/) is now supported in regular builds of coroutines conditionally depending on whether `kotlin.native.binary.memoryModel` is enabled (#2914). +* Introduced `CopyableThreadContextElement` for mutable context elements shared among multiple coroutines. Thanks @yorickhenning! (#2893). +* `transformWhile`, `awaitClose`, `ProducerScope`, `merge`, `runningFold`, `runingReduce`, and `scan` are promoted to stable API (#2971). +* `SharedFlow.subscriptionCount` no longer conflates incoming updates and gives all subscribers a chance to observe a short-lived subscription (#2488, #2863, #2871). +* `Flow` exception transparency mechanism is improved to be more exception-friendly (#3017, #2860). +* Cancellation from `flat*` operators that leverage multiple coroutines is no longer propagated upstream (#2964). +* `SharedFlow.collect` now returns `Nothing` (#2789, #2502). +* `DisposableHandle` is now `fun interface`, and corresponding inline extension is removed (#2790). +* `FlowCollector` is now `fun interface`, and corresponding inline extension is removed (#3047). +* Deprecation level of all previously deprecated signatures is raised (#3024). +* The version file is shipped with each JAR as a resource (#2941). +* Unhandled exceptions on K/N are passed to the standard library function `processUnhandledException` (#2981). +* A direct executor is used for `Task` callbacks in `kotlinx-coroutines-play-services` (#2990). +* Metadata of coroutines artifacts leverages Gradle platform to have all versions of dependencies aligned (#2865). +* Default `CoroutineExceptionHandler` is loaded eagerly and does not invoke `ServiceLoader` on its exception-handling path (#2552). +* Fixed the R8 rules for `ServiceLoader` optimization (#2880). +* Fixed BlockHound integration false-positives (#2894, #2866, #2937). +* Fixed the exception handler being invoked several times on Android, thanks to @1zaman (#3056). +* `SendChannel.trySendBlocking` is now available on Kotlin/Native (#3064). +* The exception recovery mechanism now uses `ClassValue` when available (#2997). +* JNA is updated to 5.9.0 to support Apple M1 (#3001). +* Obsolete method on internal `Delay` interface is deprecated (#2979). +* Support of deprecated `CommonPool` is removed. +* `@ExperimentalTime` is no longer needed for methods that use `Duration` (#3041). +* JDK 1.6 is no longer required for building the project (#3043). +* New version of Dokka is used, fixing the memory leak when building the coroutines and providing brand new reference visuals (https://kotlinlang.org/api/kotlinx.coroutines/) (#3051, #3054). + +### Changelog relative to version 1.6.0-RC3 + +* Restored MPP binary compatibility on K/JS and K/N (#3104). +* Fixed Dispatchers.Main not being fully initialized on Android and Swing (#3101). + +## Version 1.6.0-RC3 + +* Fixed the error in 1.6.0-RC2 because of which `Flow.collect` couldn't be called due to the `@InternalCoroutinesApi` annotation (#3082) +* Fixed some R8 warnings introduced in 1.6.0-RC (#3090) +* `TestCoroutineScheduler` now provides a `TimeSource` with its virtual time via the `timeSource` property. Thanks @hfhbd! (#3087) + +## Version 1.6.0-RC2 + +* `@ExperimentalTime` is no longer needed for methods that use `Duration` (#3041). +* `FlowCollector` is now `fun interface`, and corresponding inline extension is removed (#3047). +* Fixed the exception handler being invoked several times on Android, thanks to @1zaman (#3056). +* The deprecated `TestCoroutineScope` is no longer sealed, to simplify migration from it (#3072). +* `runTest` gives more informative errors when it times out waiting for external completion (#3071). +* `SendChannel.trySendBlocking` is now available on Kotlin/Native (#3064). +* Fixed the bug due to which `Dispatchers.Main` was not used for `delay` and `withTimeout` (#3046). +* JDK 1.6 is no longer required for building the project (#3043). +* New version of Dokka is used, fixing the memory leak when building the coroutines and providing brand new reference visuals (https://kotlinlang.org/api/kotlinx.coroutines/) (#3051, #3054). + +## Version 1.6.0-RC + +### kotlinx-coroutines-test rework + +* `kotlinx-coroutines-test` became a multiplatform library usable from K/JVM, K/JS, and K/N. +* Its API was completely reworked to address long-standing issues with consistency, structured concurrency and correctness (#1203, #1609, #2379, #1749, #1204, #1390, #1222, #1395, #1881, #1910, #1772, #1626, #1742, #2082, #2102, #2405, #2462 + ). +* The old API is deprecated for removal, but the new API is based on the similar concepts ([README](kotlinx-coroutines-test/README.md)), and the migration path is designed to be graceful: [migration guide](kotlinx-coroutines-test/MIGRATION.md) + +### Dispatchers + +* Introduced `CoroutineDispatcher.limitedParallelism` that allows obtaining a view of the original dispatcher with limited parallelism (#2919). +* `Dispatchers.IO.limitedParallelism` usages ignore the bound on the parallelism level of `Dispatchers.IO` itself to avoid starvation (#2943). +* Introduced new `Dispatchers.shutdown` method for containerized environments (#2558). +* `newSingleThreadContext` and `newFixedThreadPoolContext` are promoted to delicate API (#2919). + +### Breaking changes + +* When racing with cancellation, the `future` builder no longer reports unhandled exceptions into the global `CoroutineExceptionHandler`. Thanks @vadimsemenov! (#2774, #2791). +* `Mutex.onLock` is deprecated for removal (#2794). +* `Dispatchers.Main` is now used as the default source of time for `delay` and `withTimeout` when present(#2972). + * To opt-out from this behaviour, `kotlinx.coroutines.main.delay` system property can be set to `false`. +* Java target of coroutines build is now 8 instead of 6 (#1589). + +### Bug fixes and improvements + +* Kotlin is updated to 1.6.0. +* Kotlin/Native [new memory model](https://blog.jetbrains.com/kotlin/2021/08/try-the-new-kotlin-native-memory-manager-development-preview/) is now supported in regular builds of coroutines conditionally depending on whether `kotlin.native.binary.memoryModel` is enabled (#2914). +* Introduced `CopyableThreadContextElement` for mutable context elements shared among multiple coroutines. Thanks @yorickhenning! (#2893). +* `transformWhile`, `awaitClose`, `ProducerScope`, `merge`, `runningFold`, `runingReduce`, and `scan` are promoted to stable API (#2971). +* `SharedFlow.subscriptionCount` no longer conflates incoming updates and gives all subscribers a chance to observe a short-lived subscription (#2488, #2863, #2871). +* `Flow` exception transparency mechanism is improved to be more exception-friendly (#3017, #2860). +* Cancellation from `flat*` operators that leverage multiple coroutines is no longer propagated upstream (#2964). +* `SharedFlow.collect` now returns `Nothing` (#2789, #2502). +* `DisposableHandle` is now `fun interface`, and corresponding inline extension is removed (#2790). +* Deprecation level of all previously deprecated signatures is raised (#3024). +* The version file is shipped with each JAR as a resource (#2941). +* Unhandled exceptions on K/N are passed to the standard library function `processUnhandledException` (#2981). +* A direct executor is used for `Task` callbacks in `kotlinx-coroutines-play-services` (#2990). +* Metadata of coroutines artifacts leverages Gradle platform to have all versions of dependencies aligned (#2865). +* Default `CoroutineExceptionHandler` is loaded eagerly and does not invoke `ServiceLoader` on its exception-handling path (#2552). +* Fixed the R8 rules for `ServiceLoader` optimization (#2880). +* Fixed BlockHound integration false-positives (#2894, #2866, #2937). +* The exception recovery mechanism now uses `ClassValue` when available (#2997). +* JNA is updated to 5.9.0 to support Apple M1 (#3001). +* Obsolete method on internal `Delay` interface is deprecated (#2979). +* Support of deprecated `CommonPool` is removed. + +## Version 1.5.2 + +* Kotlin is updated to 1.5.30. +* New native targets for Apple Silicon are introduced. +* Fixed a bug when `onUndeliveredElement` was incorrectly called on a properly received elements on JS (#2826). +* Fixed `Dispatchers.Default` on React Native, it now fully relies on `setTimeout` instead of stub `process.nextTick`. Thanks to @Legion2 (#2843). +* Optimizations of `Mutex` implementation (#2581). +* `Mutex` implementation is made completely lock-free as stated (#2590). +* Various documentation and guides improvements. Thanks to @MasoodFallahpoor and @Pihanya. + +## Version 1.5.1 + +* Atomic `update`, `getAndUpdate`, and `updateAndGet` operations of `MutableStateFlow` (#2720). +* `Executor.asCoroutineDispatcher` implementation improvements (#2601): + * If the target executor is `ScheduledExecutorService`, then its `schedule` API is used for time-related coroutine operations. + * `RemoveOnCancelPolicy` is now part of the public contract. +* Introduced overloads for `Task.asDeferred` and `Task.await` that accept `CancellationTokenSource` for bidirectional cancellation (#2527). +* Reactive streams are updated to `1.0.3` (#2740). +* `CopyableThrowable` is allowed to modify the exception message during stacktrace recovery (#1931). +* `CoroutineDispatcher.releaseInterceptedContinuation` is now a `final` method (#2785). +* Closing a Handler underlying `Handler.asCoroutineDispatcher` now causes the dispatched coroutines to be canceled on `Dispatchers.IO (#2778)`. +* Kotlin is updated to 1.5.20. +* Fixed a spurious `ClassCastException` in `releaseInterceptedContinuation` and `IllegalStateException` from `tryReleaseClaimedContinuation` (#2736, #2768). +* Fixed inconsistent exception message during stacktrace recovery for non-suspending channel iterators (#2749). +* Fixed linear stack usage for `CompletableFuture.asDeferred` when the target future has a long chain of listeners (#2730). +* Any exceptions from `CoroutineDispatcher.isDispatchNeeded` are now considered as fatal and are propagated to the caller (#2733). +* Internal `DebugProbesKt` (used in the debugger implementation) are moved from `debug` to `core` module. + +## Version 1.5.0 + +Note that this is a full changelog relative to 1.4.3 version. Changelog relative to 1.5.0-RC can be found in the end. + +### Channels API + +* Major channels API rework (#330, #974). Existing `offer`, `poll`, and `sendBlocking` methods are deprecated, internal `receiveCatching` and `onReceiveCatching` removed, `receiveOrNull` and `onReceiveOrNull` are completely deprecated. Previously deprecated `SendChannel.isFull` declaration is removed. Channel operators deprecated with `ERROR` are now `HIDDEN`. +* New methods `receiveCatching`, `onReceiveCatching` `trySend`, `tryReceive`, and `trySendBlocking` along with the new result type `ChannelResult` are introduced. They provide better type safety, are less error-prone, and have a consistent future-proof naming scheme. The full rationale behind this change can be found [here](https://github.com/Kotlin/kotlinx.coroutines/issues/974#issuecomment-806569582). +* `BroadcastChannel` and `ConflatedBroadcastChannel` are marked as `ObsoleteCoroutinesApi` in the favor or `SharedFlow` and `StateFlow`. The migration scheme can be found in their documentation. These classes will be deprecated in the next major release. +* `callbackFlow` and `channelFlow` are promoted to stable API. + +### Reactive integrations + +* All existing API in modules `kotlinx-coroutines-rx2`, `kotlinx-coroutines-rx3`, `kotlinx-coroutines-reactive`, `kotlinx-coroutines-reactor`, and `kotlinx-coroutines-jdk9` were revisited and promoted to stable (#2545). +* `publish` is no longer allowed to emit `null` values (#2646). +* Misleading `awaitSingleOr*` functions on `Publisher` type are deprecated (#2591). +* `MaybeSource.await` is deprecated in the favor of `awaitSingle`, additional lint functions for `Mono` are added in order to prevent ambiguous `Publisher` usages (#2628, #1587). +* `ContextView` support in `kotlinx-coroutines-reactor` (#2575). +* All reactive builders no longer ignore inner cancellation exceptions preventing their completion (#2262, #2646). +* `MaybeSource.collect` and `Maybe.collect` properly finish when they are completed without a value (#2617). +* All exceptions are now consistently handled according to reactive specification, whether they are considered 'fatal' or not by reactive frameworks (#2646). + +### Other improvements + +* Kotlin version is upgraded to 1.5.0 and JVM target is updated to 1.8. +* `Flow.last` and `Flow.lastOrNull` operators (#2246). +* `Flow.runningFold` operator (#2641). +* `CoroutinesTimeout` rule for JUnit5 (#2197). +* Internals of `Job` and `AbstractCoroutine` was reworked, resulting in smaller code size, less memory footprint, and better performance (#2513, #2512). +* `CancellationException` from Kotlin standard library is used for cancellation on Kotlin/JS and Kotlin/Native (#2638). +* Introduced new `DelicateCoroutinesApi` annotation that warns users about potential target API pitfalls and suggests studying API's documentation first. The only delicate API right now is `GlobalScope` (#2637). +* Fixed bug introduced in `1.4.3` when `kotlinx-coroutines-core.jar` triggered IDEA debugger failure (#2619). +* Fixed memory leak of `ChildHandlerNode` with reusable continuations (#2564). +* Various documentation improvements (#2555, #2589, #2592, #2583, #2437, #2616, #2633, #2560). + +### Changelog relative to version 1.5.0-RC + +* Fail-fast during `emitAll` called from cancelled `onCompletion` operator (#2700). +* Flows returned by `stateIn`/`shareIn` keep strong reference to sharing job (#2557). +* Rename internal `TimeSource` to `AbstractTimeSource` due to import issues (#2691). +* Reverted the change that triggered IDEA coroutines debugger crash (#2695, reverted #2291). +* `watchosX64` target support for Kotlin/Native (#2524). +* Various documentation fixes and improvements. + +## Version 1.5.0-RC + +### Channels API + +* Major channels API rework (#330, #974). Existing `offer`, `poll`, and `sendBlocking` methods are deprecated, internal `receiveCatching` and `onReceiveCatching` removed, `receiveOrNull` and `onReceiveOrNull` are completely deprecated. Previously deprecated `SendChannel.isFull` declaration is removed. Channel operators deprecated with `ERROR` are now `HIDDEN`. +* New methods `receiveCatching`, `onReceiveCatching` `trySend`, `tryReceive`, and `trySendBlocking` along with the new result type `ChannelResult` are introduced. They provide better type safety, are less error-prone, and have a consistent future-proof naming scheme. The full rationale behind this change can be found [here](https://github.com/Kotlin/kotlinx.coroutines/issues/974#issuecomment-806569582). +* `BroadcastChannel` and `ConflatedBroadcastChannel` are marked as `ObsoleteCoroutinesApi` in the favor or `SharedFlow` and `StateFlow`. The migration scheme can be found in their documentation. These classes will be deprecated in the next major release. +* `callbackFlow` and `channelFlow` are promoted to stable API. + +### Reactive integrations + +* All existing API in modules `kotlinx-coroutines-rx2`, `kotlinx-coroutines-rx3`, `kotlinx-coroutines-reactive`, `kotlinx-coroutines-reactor`, and `kotlinx-coroutines-jdk9` were revisited and promoted to stable (#2545). +* `publish` is no longer allowed to emit `null` values (#2646). +* Misleading `awaitSingleOr*` functions on `Publisher` type are deprecated (#2591). +* `MaybeSource.await` is deprecated in the favor of `awaitSingle`, additional lint functions for `Mono` are added in order to prevent ambiguous `Publisher` usages (#2628, #1587). +* `ContextView` support in `kotlinx-coroutines-reactor` (#2575). +* All reactive builders no longer ignore inner cancellation exceptions preventing their completion (#2262, #2646). +* `MaybeSource.collect` and `Maybe.collect` properly finish when they are completed without a value (#2617). +* All exceptions are now consistently handled according to reactive specification, whether they are considered 'fatal' or not by reactive frameworks (#2646). + +### Other improvements + +* `Flow.last` and `Flow.lastOrNull` operators (#2246). +* `Flow.runningFold` operator (#2641). +* `CoroutinesTimeout` rule for JUnit5 (#2197). +* Internals of `Job` and `AbstractCoroutine` was reworked, resulting in smaller code size, less memory footprint, and better performance (#2513, #2512). +* `CancellationException` from Kotlin standard library is used for cancellation on Kotlin/JS and Kotlin/Native (#2638). +* Introduced new `DelicateCoroutineApi` annotation that warns users about potential target API pitfalls and suggests studying API's documentation first. The only delicate API right now is `GlobalScope` (#2637). +* Fixed bug introduced in `1.4.3` when `kotlinx-coroutines-core.jar` triggered IDEA debugger failure (#2619). +* Fixed memory leak of `ChildHandlerNode` with reusable continuations (#2564). +* Various documentation improvements (#2555, #2589, #2592, #2583, #2437, #2616, #2633, #2560). + +## Version 1.4.3 + +### General changes + +* Thread context is properly preserved and restored for coroutines without `ThreadContextElement` (#985) +* `ThreadContextElement`s are now restored in the opposite order from update (#2195) +* Improved performance of combine with 4 parameters, thanks to @alexvanyo (#2419) +* Debug agent sanitizer leaves at least one frame with source location (#1437) +* Update Reactor version in `kotlinx-coroutines-reactor` to `3.4.1`, thanks to @sokomishalov (#2432) +* `callInPlace` contract added to `ReceiveChannel.consume` (#941) +* `CoroutineStart.UNDISPATCHED` promoted to stable API (#1393) +* Kotlin updated to 1.4.30 +* `kotlinx.coroutines` are now released directly to MavenCentral +* Reduced the size of `DispatchedCoroutine` by a field +* Internal class `TimeSource` renamed to `SchedulerTimeSource` to prevent wildcard import issues (#2537) + +### Bug fixes + +* Fixed the problem that prevented implementation via delegation for `Job` interface (#2423) +* Fixed incorrect ProGuard rules that allowed shrinking volatile felds (#1564) +* Fixed `await`/`asDeferred` for `MinimalStage` implementations in jdk8 module (#2456) +* Fixed bug when `onUndeliveredElement` wasn't called for unlimited channels (#2435) +* Fixed a bug when `ListenableFuture.isCancelled` returned from `asListenableFuture` could have thrown an exception, thanks to @vadimsemenov (#2421) +* Coroutine in `callbackFlow` and `produce` is properly cancelled when the channel was closed separately (#2506) + +## Version 1.4.2 + +* Fixed `StackOverflowError` in `Job.toString` when `Job` is observed in its intermediate state (#2371). +* Improved liveness and latency of `Dispatchers.Default` and `Dispatchers.IO` in low-loaded mode (#2381). +* Improved performance of consecutive `Channel.cancel` invocations (#2384). +* `SharingStarted` is now `fun` interface (#2397). +* Additional lint settings for `SharedFlow` to catch programmatic errors early (#2376). +* Fixed bug when mutex and semaphore were not released during cancellation (#2390, thanks to @Tilps for reproducing). +* Some corner cases in cancellation propagation between coroutines and listenable futures are repaired (#1442, thanks to @vadimsemenov). +* Fixed unconditional cast to `CoroutineStackFrame` in exception recovery that triggered failures of instrumented code (#2386). +* Platform-specific dependencies are removed from `kotlinx-coroutines-javafx` (#2360). + +## Version 1.4.1 + +This is a patch release with an important fix to the `SharedFlow` implementation. + +* SharedFlow: Fix scenario with concurrent emitters and cancellation of subscriber (#2359, thanks to @vehovsky for the bug report). + +## Version 1.4.0 + +### Improvements + +* `StateFlow`, `SharedFlow` and corresponding operators are promoted to stable API (#2316). +* `Flow.debounce` operator with timeout selector based on each individual element is added (#1216, thanks to @mkano9!). +* `CoroutineContext.job` extension property is introduced (#2159). +* `Flow.combine operator` is reworked: + * Complete fairness is maintained for single-threaded dispatchers. + * Its performance is improved, depending on the use-case, by at least 50% (#2296). + * Quadratic complexity depending on the number of upstream flows is eliminated (#2296). + * `crossinline` and `inline`-heavy internals are removed, fixing sporadic SIGSEGV on Mediatek Android devices (#1683, #1743). +* `Flow.zip` operator performance is improved by 40%. +* Various API has been promoted to stable or its deprecation level has been raised (#2316). + +### Bug fixes + +* Suspendable `stateIn` operator propagates exception to the caller when upstream fails to produce initial value (#2329). +* Fix `SharedFlow` with replay for subscribers working at different speed (#2325). +* Do not fail debug agent installation when security manager does not provide access to system properties (#2311). +* Cancelled lazy coroutines are properly cleaned up from debug agent output (#2294). +* `BlockHound` false-positives are correctly filtered out (#2302, #2190, #2303). +* Potential crash during a race between cancellation and upstream in `Observable.asFlow` is fixed (#2104, #2299, thanks to @LouisCAD and @drinkthestars). + +## Version 1.4.0-M1 + +### Breaking changes + +* The concept of atomic cancellation in channels is removed. All operations in channels + and corresponding `Flow` operators are cancellable in non-atomic way (#1813). +* If `CoroutineDispatcher` throws `RejectedExecutionException`, cancel current `Job` and schedule its execution to `Dispatchers.IO` (#2003). +* `CancellableContinuation.invokeOnCancellation` is invoked if the continuation was cancelled while its resume has been dispatched (#1915). +* `Flow.singleOrNull` operator is aligned with standard library and does not longer throw `IllegalStateException` on multiple values (#2289). + +### New experimental features + +* `SharedFlow` primitive for managing hot sources of events with support of various subscription mechanisms, replay logs and buffering (#2034). +* `Flow.shareIn` and `Flow.stateIn` operators to transform cold instances of flow to hot `SharedFlow` and `StateFlow` respectively (#2047). + +### Other + +* Support leak-free closeable resources transfer via `onUndeliveredElement` in channels (#1936). +* Changed ABI in reactive integrations for Java interoperability (#2182). +* Fixed ProGuard rules for `kotlinx-coroutines-core` (#2046, #2266). +* Lint settings were added to `Flow` to avoid accidental capturing of outer `CoroutineScope` for cancellation check (#2038). + +### External contributions + +* Allow nullable types in `Flow.firstOrNull` and `Flow.singleOrNull` by @ansman (#2229). +* Add `Publisher.awaitSingleOrDefault|Null|Else` extensions by @sdeleuze (#1993). +* `awaitCancellation` top-level function by @LouisCAD (#2213). +* Significant part of our Gradle build scripts were migrated to `.kts` by @turansky. + +Thank you for your contributions and participation in the Kotlin community! + +## Version 1.3.9 + +* Support of `CoroutineContext` in `Flow.asPublisher` and similar reactive builders (#2155). +* Kotlin updated to 1.4.0. +* Transition to new HMPP publication scheme for multiplatform usages: + * Artifacts `kotlinx-coroutines-core-common` and `kotlinx-coroutines-core-native` are removed. + * For multiplatform usages, it's enough to [depend directly](README.md#multiplatform) on `kotlinx-coroutines-core` in `commonMain` source-set. + * The same artifact coordinates can be used to depend on platform-specific artifact in platform-specific source-set. + +## Version 1.3.8 + +### New experimental features + +* Added `Flow.transformWhile operator` (#2065). +* Replaced `scanReduce` with `runningReduce` to be consistent with the Kotlin standard library (#2139). + +### Bug fixes and improvements + +* Improve user experience for the upcoming coroutines debugger (#2093, #2118, #2131). +* Debugger no longer retains strong references to the running coroutines (#2129). +* Fixed race in `Flow.asPublisher` (#2109). +* Fixed `ensureActive` to work in the empty context case to fix `IllegalStateException` when using flow from `suspend fun main` (#2044). +* Fixed a problem with `AbortFlowException` in the `Flow.first` operator to avoid erroneous `NoSuchElementException` (#2051). +* Fixed JVM dependency on Android annotations (#2075). +* Removed keep rules mentioning `kotlinx.coroutines.android` from core module (#2061 by @mkj-gram). +* Corrected some docs and examples (#2062, #2071, #2076, #2107, #2098, #2127, #2078, #2135). +* Improved the docs and guide on flow cancellation (#2043). +* Updated Gradle version to `6.3` (it only affects multiplatform artifacts in this release). + +## Version 1.3.7 + +* Fixed problem that triggered Android Lint failure (#2004). +* New `Flow.cancellable()` operator for cooperative cancellation (#2026). +* Emissions from `flow` builder now check cancellation status and are properly cancellable (#2026). +* New `currentCoroutineContext` function to use unambiguously in the contexts with `CoroutineScope` in receiver position (#2026). +* `EXACTLY_ONCE` contract support in coroutine builders. +* Various documentation improvements. + +## Version 1.3.6 + +### Flow + +* `StateFlow`, new primitive for state handling (#1973, #1816, #395). The `StateFlow` is designed to eventually replace `ConflatedBroadcastChannel` for state publication scenarios. Please, try it and share your feedback. Note, that Flow-based primitives to publish events will be added later. For events you should continue to either use `BroadcastChannel(1)`, if you put events into the `StateFlow`, protect them from double-processing with flags. +* `Flow.onEmpty` operator is introduced (#1890). +* Behavioural change in `Flow.onCompletion`, it is aligned with `invokeOnCompletion` now and passes `CancellationException` to its cause parameter (#1693). +* A lot of Flow operators have left its experimental status and are promoted to stable API. + +### Other + +* `runInterruptible` primitive to tie cancellation with thread interruption for blocking calls. Contributed by @jxdabc (#1947). +* Integration module with RxJava3 is introduced. Contributed by @ZacSweers (#1883) +* Integration with [BlockHound](https://github.com/reactor/BlockHound) in `kotlinx-coroutines-debug` module (#1821, #1060). +* Memory leak in ArrayBroadcastChannel is fixed (#1885). +* Behavioural change in `suspendCancellableCoroutine`, cancellation is established before invoking passed block argument (#1671). +* Debug agent internals are moved into `kotlinx-coroutines-core` for better integration with IDEA. It should not affect library users and all the redundant code should be properly eliminated with R8. +* ClassCastException with reusable continuations bug is fixed (#1966). +* More precise scheduler detection for `Executor.asCoroutineDispatcher` (#1992). +* Kotlin updated to 1.3.71. + +## Version 1.3.5 + +* `firstOrNull` operator. Contributed by @bradynpoulsen. +* `java.time` adapters for Flow operators. Contributed by @fvasco. +* `kotlin.time.Duration` support (#1402). Contributed by @fvasco. +* Memory leak with a mix of reusable and non-reusable continuations is fixed (#1855). +* `DebugProbes` are ready for production installation: its performance is increased, the flag to disable creation stacktraces to reduce the footprint is introduced (#1379, #1372). +* Stacktrace recovery workaround for Android 6.0 and earlier bug (#1866). +* New integration module: `kotlinx-coroutines-jdk9` with adapters for `java.util.concurrent.Flow`. +* `BroadcastChannel.close` properly starts lazy coroutine (#1713). +* `kotlinx-coroutines-bom` is published without Gradle metadata. +* Make calls to service loader in reactor integrations optimizable by R8 (#1817). + +## Version 1.3.4 + +### Flow + +* Detect missing `awaitClose` calls in `callbackFlow` to make it less error-prone when used with callbacks (#1762, #1770). This change makes `callbackFlow` **different** from `channelFlow`. +* `ReceiveChannel.asFlow` extension is introduced (#1490). +* Enforce exception transparency invariant in `flow` builder (#1657). +* Proper `Dispatcher` support in `Flow` reactive integrations (#1765). +* Batch `Subscription.request` calls in `Flow` reactive integration (#766). +* `ObservableValue.asFlow` added to JavaFx integration module (#1695). +* `ObservableSource.asFlow` added to RxJava2 integration module (#1768). + +### Other changes + +* `kotlinx-coroutines-core` is optimized for R8, making it much smaller for Android usages (75 KB for `1.3.4` release). +* Performance of `Dispatchers.Default` is improved (#1704, #1706). +* Kotlin is updated to 1.3.70. +* `CoroutineDispatcher` and `ExecutorCoroutineDispatcher` experimental coroutine context keys are introduced (#1805). +* Performance of various `Channel` operations is improved (#1565). + +## Version 1.3.3 + +### Flow +* `Flow.take` performance is significantly improved (#1538). +* `Flow.merge` operator (#1491). +* Reactive Flow adapters are promoted to stable API (#1549). +* Reusable cancellable continuations were introduced that improved the performance of various flow operators and iteration over channels (#1534). +* Fixed interaction of multiple flows with `take` operator (#1610). +* Throw `NoSuchElementException` instead of `UnsupportedOperationException` for empty `Flow` in `reduce` operator (#1659). +* `onCompletion` now rethrows downstream exceptions on emit attempt (#1654). +* Allow non-emitting `withContext` from `flow` builder (#1616). + +### Debugging + +* `DebugProbes.dumpCoroutines` is optimized to be able to print the 6-digit number of coroutines (#1535). +* Properly capture unstarted lazy coroutines in debugger (#1544). +* Capture coroutines launched from within a test constructor with `CoroutinesTimeout` test rule (#1542). +* Stacktraces of `Job`-related coroutine machinery are shortened and prettified (#1574). +* Stacktrace recovery unification that should provide a consistent experience recover of stacktrace (#1597). +* Stacktrace recovery for `withTimeout` is supported (#1625). +* Do not recover exception with a single `String` parameter constructor that is not a `message` (#1631). + +### Other features + +* `Dispatchers.Default` and `Dispatchers.IO` rework: CPU consumption is significantly lower, predictable idle threads termination (#840, #1046, #1286). +* Avoid `ServiceLoader` for loading `Dispatchers.Main` (#1572, #1557, #878, #1606). +* Consistently handle undeliverable exceptions in RxJava and Reactor integrations (#252, #1614). +* `yield` support in immediate dispatchers (#1474). +* `CompletableDeferred.completeWith(result: Result<T>)` is introduced. +* Added support for tvOS and watchOS-based Native targets (#1596). + +### Bug fixes and improvements + +* Kotlin version is updated to 1.3.61. +* `CoroutineDispatcher.isDispatchNeeded` is promoted to stable API (#1014). +* Livelock and stackoverflows in mutual `select` expressions are fixed (#1411, #504). +* Properly handle `null` values in `ListenableFuture` integration (#1510). +* Making ReceiveChannel.cancel linearizability-friendly. +* Linearizability of Channel.close in a complex contended cases (#1419). +* ArrayChannel.isBufferEmpty atomicity is fixed (#1526). +* Various documentation improvements. +* Reduced bytecode size of `kotlinx-coroutines-core`, reduced size of minified `dex` when using basic functionality of `kotlinx-coroutines`. + +## Version 1.3.2 + +This is a maintenance release that does not include any new features or bug fixes. + +* Reactive integrations for `Flow` are promoted to stable API. +* Obsolete reactive API is deprecated. +* Deprecation level for API deprecated in 1.3.0 is increased. +* Various documentation improvements. + +## Version 1.3.1 + +This is a minor update with various fixes: +* Flow: Fix recursion in combineTransform<T1, T2, R> (#1466). +* Fixed race in the Semaphore (#1477). +* Repaired some of ListenableFuture.kt's cancellation corner cases (#1441). +* Consistently unwrap exception in slow path of CompletionStage.asDeferred (#1479). +* Various fixes in documentation (#1496, #1476, #1470, #1468). +* Various cleanups and additions in tests. + +Note: Kotlin/Native artifacts are now published with Gradle metadata format version 1.0, so you will need +Gradle version 5.3 or later to use this version of kotlinx.coroutines in your Kotlin/Native project. + +## Version 1.3.0 + +### Flow + +This version is the first stable release with [`Flow`](https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.flow/-flow/index.html) API. + +All `Flow` API not marked with `@FlowPreview` or `@ExperimentalCoroutinesApi` annotations are stable and here to stay. +Flow declarations marked with `@ExperimentalCoroutinesApi` have [the same guarantees](/docs/topics/compatibility.md#experimental-api) as regular experimental API. +Please note that API marked with `@FlowPreview` have [weak guarantees](/docs/topics/compatibility.md#flow-preview-api) on source, binary and semantic compatibility. + +### Changelog + +* A new [guide section](/docs/topics/flow.md) about Flow. +* `CoroutineDispatcher.asExecutor` extension (#1450). +* Fixed bug when `select` statement could report the same exception twice (#1433). +* Fixed context preservation in `flatMapMerge` in a case when collected values were immediately emitted to another flow (#1440). +* Reactive Flow integrations enclosing files are renamed for better interoperability with Java. +* Default buffer size in all Flow operators is increased to 64. +* Kotlin updated to 1.3.50. + +## Version 1.3.0-RC2 + +### Flow improvements +* Operators for UI programming are reworked for the sake of consistency, naming scheme for operator overloads is introduced: + * `combineLatest` is deprecated in the favor of `combine`. + * `combineTransform` operator for non-trivial transformations (#1224). + * Top-level `combine` and `combineTransform` overloads for multiple flows (#1262). + * `switchMap` is deprecated. `flatMapLatest`, `mapLatest` and `transformLatest` are introduced instead (#1335). + * `collectLatest` terminal operator (#1269). + +* Improved cancellation support in `flattenMerge` (#1392). +* `channelFlow` cancellation does not leak to the parent (#1334). +* Fixed flow invariant enforcement for `suspend fun main` (#1421). +* `delayEach` and `delayFlow` are deprecated (#1429). + +### General changes +* Integration with Reactor context + * Propagation of the coroutine context of `await` calls into Mono/Flux builder. + * Publisher.asFlow propagates coroutine context from `collect` call to the Publisher. + * New `Flow.asFlux ` builder. + +* ServiceLoader-code is adjusted to avoid I/O on the Main thread on newer (3.6.0+) Android toolchain. +* Stacktrace recovery support for minified builds on Android (#1416). +* Guava version in `kotlinx-coroutines-guava` updated to `28.0`. +* `setTimeout`-based JS dispatcher for platforms where `process` is unavailable (#1404). +* Native, JS and common modules are added to `kotlinx-coroutines-bom`. +* Fixed bug with ignored `acquiredPermits` in `Semaphore` (#1423). + +## Version 1.3.0-RC + +### Flow + +* Core `Flow` API is promoted to stable +* New basic `Flow` operators: `withIndex`, `collectIndexed`, `distinctUntilChanged` overload +* New core `Flow` operators: `onStart` and `onCompletion` +* `ReceiveChannel.consumeAsFlow` and `emitAll` (#1340) + +### General changes + +* Kotlin updated to 1.3.41 +* Added `kotlinx-coroutines-bom` with Maven Bill of Materials (#1110) +* Reactive integrations are seriously improved + * All builders now are top-level functions instead of extensions on `CoroutineScope` and prohibit `Job` instance in their context to simplify lifecycle management + * Fatal exceptions are handled consistently (#1297) + * Integration with Reactor Context added (#284) +* Stacktrace recovery for `suspend fun main` (#1328) +* `CoroutineScope.cancel` extension with message (#1338) +* Protection against non-monotonic clocks in `delay` (#1312) +* `Duration.ZERO` is handled properly in JDK 8 extensions (#1349) +* Library code is adjusted to be more minification-friendly + +## Version 1.3.0-M2 + + * Kotlin updated to 1.3.40. + * `Flow` exception transparency concept. + * New declarative `Flow` operators: `onCompletion`, `catch`, `retryWhen`, `launchIn`. `onError*` operators are deprecated in favour of `catch`. (#1263) + * `Publisher.asFlow` is integrated with `buffer` operator. + * `Publisher.openSubscription` default request size is `1` instead of `0` (#1267). + +## Version 1.3.0-M1 + +Flow: + * Core `Flow` interfaces and operators are graduated from preview status to experimental. + * Context preservation invariant rework (#1210). + * `channelFlow` and `callbackFlow` replacements for `flowViaChannel` for concurrent flows or callback-based APIs. + * `flow` prohibits emissions from non-scoped coroutines by default and recommends to use `channelFlow` instead to avoid most of the concurrency-related bugs. + * Flow cannot be implemented directly + * `AbstractFlow` is introduced for extension (e.g. for managing state) and ensures all context preservation invariants. + * Buffer size is decoupled from all operators that imply channel usage (#1233) + * `buffer` operator can be used to adjust buffer size of any buffer-dependent operator (e.g. `channelFlow`, `flowOn` and `flatMapMerge`). + * `conflate` operator is introduced. + * Flow performance is significantly improved. + * New operators: `scan`, `scanReduce`, `first`, `emitAll`. + * `flowWith` and `flowViaChannel` are deprecated. + * `retry` ignores cancellation exceptions from upstream when the flow was externally cancelled (#1122). + * `combineLatest` overloads for multiple flows (#1193). + * Fixed numerical overflow in `drop` operator. + +Channels: + * `consumeEach` is promoted to experimental API (#1080). + * Conflated channels always deliver the latest value after closing (#332, #1235). + * Non-suspending `ChannelIterator.next` to improve iteration performance (#1162). + * Channel exception types are consistent with `produce` and are no longer swallowed as cancellation exceptions in case of programmatic errors (#957, #1128). + * All operators on channels (that were prone to coroutine leaks) are deprecated in the favor of `Flow`. + +General changes: + * Kotlin updated to 1.3.31 + * `Semaphore` implementation (#1088) + * Loading of `Dispatchers.Main` is tweaked so the latest version of R8 can completely remove I/O when loading it (#1231). + * Performace of all JS dispatchers is significantly improved (#820). + * `withContext` checks cancellation status on exit to make reasoning about sequential concurrent code easier (#1177). + * Consistent exception handling mechanism for complex hierarchies (#689). + * Convenient overload for `CoroutinesTimeout.seconds` (#1184). + * Fix cancellation bug in onJoin (#1130). + * Prevent internal names clash that caused errors for ProGuard (#1159). + * POSIX's `nanosleep` as `delay` in `runBlocking ` in K/N (#1225). + +## Version 1.2.2 + +* Kotlin updated to 1.3.40. + +## Version 1.2.1 + +Major: + * Infrastructure for testing coroutine-specific code in `kotlinx-coroutines-test`: `runBlockingTest`, `TestCoroutineScope` and `TestCoroutineDispatcher`, contributed by Sean McQuillan (@objcode). Obsolete `TestCoroutineContext` from `kotlinx-coroutines-core` is deprecated. + * `Job.asCompletableFuture` extension in jdk8 module (#1113). + +Flow improvements: + * `flowViaChannel` rework: block parameter is no longer suspending, but provides `CoroutineScope` receiver and allows conflated channel (#1081, #1112). + * New operators: `switchMap`, `sample`, `debounce` (#1107). + * `consumerEach` is deprecated on `Publisher`, `ObservableSource` and `MaybeSource`, `collect` extension is introduced instead (#1080). + +Other: + * Race in Job.join and concurrent cancellation is fixed (#1123). + * Stacktrace recovery machinery improved: cycle detection works through recovered exceptions, cancellation exceptions are recovered on cancellation fast-path. + * Atomicfu-related bug fixes: publish transformed artifacts, do not propagate transitive atomicfu dependency (#1064, #1116). + * Publication to NPM fixed (#1118). + * Misplaced resources are removed from the final jar (#1131). + +## Version 1.2.0 + + * Kotlin updated to 1.3.30. + * New API: `CancellableContinuation.resume` with `onCancelling` lambda (#1044) to consistently handle closeable resources. + * Play services task version updated to 16.0.1. + * `ReceiveChannel.isEmpty` is no longer deprecated + +A lot of `Flow` improvements: + * Purity property is renamed to context preservation and became more restrictive. + * `zip` and `combineLatest` operators. + * Integration with RxJava2 + * `flatMap`, `merge` and `concatenate` are replaced with `flattenConcat`, `flattenMerge`, `flatMapConcat` and `flatMapMerge`. + * Various documentation improvements and minor bug fixes. + +Note that `Flow` **is not** leaving its [preview status](/docs/topics/compatibility.md#flow-preview-api). + +## Version 1.2.0-alpha-2 + +This release contains major [feature preview](/docs/topics/compatibility.md#flow-preview-api): cold streams aka `Flow` (#254). + +Performance: +* Performance of `Dispatcher.Main` initialization is significantly improved (#878). + +## Version 1.2.0-alpha + +* Major debug agent improvements. Real stacktraces are merged with coroutine stacktraces for running coroutines, merging heuristic is improved, API is cleaned up and is on its road to stabilization (#997). +* `CoroutineTimeout` rule or JUnit4 is introduced to simplify coroutines debugging (#938). +* Stacktrace recovery improvements. Exceptions with custom properties are no longer copied, `CopyableThrowable` interface is introduced, machinery is [documented](https://github.com/Kotlin/kotlinx.coroutines/blob/develop/docs/debugging.md) (#921, #950). +* `Dispatchers.Unconfined`, `MainCoroutineDispatcher.immediate`, `MainScope` and `CoroutineScope.cancel` are promoted to stable API (#972). +* `CompletableJob` is introduced (#971). +* Structured concurrency is integrated into futures and listenable futures (#1008). +* `ensurePresent` and `isPresent` extensions for `ThreadLocal` (#1028). +* `ensureActive` extensions for `CoroutineContext`, `CoroutineScope` and `Job` (#963). +* `SendChannel.isFull` and `ReceiveChannel.isEmpty` are deprecated (#1053). +* `withContext` checks cancellation on entering (#962). +* Operator `invoke` on `CoroutineDispatcher` (#428). +* Java 8 extensions for `delay` and `withTimeout` now properly handle too large values (#428). +* A global exception handler for fatal exceptions in coroutines is introduced (#808, #773). +* Major improvements in cancellation machinery and exceptions delivery consistency. Cancel with custom exception is completely removed. +* Kotlin version is updated to 1.3.21. +* Do not use private API on newer Androids to handle exceptions (#822). + +Bug fixes: +* Proper `select` support in debug agent (#931). +* Proper `supervisorScope` support in debug agent (#915). +* Throwing `initCause` does no longer trigger an internal error (#933). +* Lazy actors are started when calling `close` in order to cleanup their resources (#939). +* Minor bugs in reactive integrations are fixed (#1008). +* Experimental scheduler shutdown sequence is fixed (#990). + +## Version 1.1.1 + +* Maintenance release, no changes in the codebase +* Kotlin is updated to 1.3.20 +* Gradle is updated to 4.10 +* Native module is published with Gradle metadata v0.4 + +## Version 1.1.0 + +* Kotlin version updated to 1.3.11. +* Resumes to `CancellableContinuation` in the final state produce `IllegalStateException` (#901). This change does not affect #830, races between resume and cancellation do not lead to an exceptional situation. +* `runBlocking` is integrated with `Dispatchers.Unconfined` by sharing an internal event loop. This change does not affect the semantics of the previously correct code but allows to mix multiple `runBlocking` and unconfined tasks (#860). + +## Version 1.1.0-alpha + +### Major improvements in coroutines testing and debugging +* New module: [`kotlinx-coroutines-debug`](https://github.com/Kotlin/kotlinx.coroutines/blob/master/core/kotlinx-coroutines-debug/README.md). Debug agent that improves coroutines stacktraces, allows to print all active coroutines and its hierarchies and can be installed as Java agent. +* New module: [`kotlinx-coroutines-test`](https://github.com/Kotlin/kotlinx.coroutines/blob/master/core/kotlinx-coroutines-test/README.md). Allows setting arbitrary `Dispatchers.Main` implementation for tests (#810). +* Stacktrace recovery mechanism. Exceptions from coroutines are recovered from current coroutine stacktraces to simplify exception diagnostic. Enabled in debug mode, controlled by `kotlinx.coroutines.debug` system property (#493). + +### Other improvements +* `MainScope` factory and `CoroutineScope.cancel` extension (#829). One line `CoroutineScope` integration! +* `CancellableContinuation` race between `resumeWithException` and `cancel` is addressed, exceptions during cancellation are no longer reported to exception handler (#830, #892). +* `Dispatchers.Default` now consumes much less CPU on JVM (#840). +* Better diagnostic and fast failure if an uninitialized dispatcher is used (#880). +* Conflated channel becomes linearizable. +* Fixed inconsistent coroutines state when the result of the coroutine had type `DisposableHandle` (#835). +* Fixed `JavaFx` initialization bug (#816). +* `TimeoutCancellationException` is thrown by `withTimeout` instead of `CancellationException` if negative timeout is supplied (#870). +* Kotlin/Native single-threaded workers support: coroutines can be safely used in multiple independent K/N workers. +* jsdom support in `Dispatchers.Default` on JS. +* rxFlowable generic parameter is now restricted with Any. +* Guava 27 support in `kotlinx-coroutines-guava`. +* Coroutines are now built with progressive mode. +* Various fixes in the documentation. + +## Version 1.0.1 + +* Align `publisher` implementation with Reactive TCK. +* Reimplement `future` coroutine builders on top of `AbstractCoroutine` (#751). +* Performance optimizations in `Dispatchers.Default` and `Dispatchers.IO`. +* Use only public API during `JavaFx` instantiation, fixes warnings on Java 9 and build on Java 11 (#463). +* Updated contract of `CancellableContinuation.resumeWithException` (documentation fix, see #712). +* Check cancellation on fast-path of all in-place coroutine builders (`withContext`, `coroutineScope`, `supervisorScope`, `withTimeout` and `withTimeoutOrNull`). +* Add optional prefix to thread names of `ExperimentalCoroutineDispatcher` (#661). +* Fixed bug when `ExperimentalCoroutineDispatcher` could end up in inconsistent state if `Thread` constructor throws an exception (#748). + +## Version 1.0.0 + +* All Kotlin dependencies updated to 1.3 release version. +* Fixed potential memory leak in `HandlerDispatcher.scheduleResumeAfterDelay`, thanks @cbeyls. +* `yield` support for `Unconfined` and immediate dispatchers (#737). +* Various documentation improvements. + +## Version 1.0.0-RC1 + +* Coroutines API is updated to Kotlin 1.3. +* Deprecated API is removed or marked as `internal`. +* Experimental and internal coroutine API is marked with corresponding `kotlin.experimental.Experimental` annotation. If you are using `@ExperimentalCoroutinesApi` or `@InternalCoroutinesApi` you should explicitly opt-in, otherwise compilation warning (or error) will be produced. +* `Unconfined` dispatcher (and all dispatchers which support immediate invocation) forms event-loop on top of current thread, thus preventing all `StackOverflowError`s. `Unconfined` dispatcher is now much safer for the general use and may leave its experimental status soon (#704). +* Significantly improved performance of suspending hot loops in `kotlinx.coroutines` (#537). +* Proguard rules are embedded into coroutines JAR to assist jettifier (#657) +* Fixed bug in shutdown sequence of `runBlocking` (#692). +* `ReceiveChannel.receiveOrNull` is marked as obsolete and deprecated. +* `Job.cancel(cause)` and `ReceiveChannel.cancel(cause)` are deprecated, `cancel()` returns `Unit` (#713). + +## Version 0.30.2 + +* `Dispatchers.Main` is instantiated lazily (see #658 and #665). +* Blocking coroutine dispatcher views are now shutdown properly (#678). +* Prevent leaking Kotlin 1.3 from atomicfu dependency (#659). +* Thread-pool based dispatcher factories are marked as obsolete (#261). +* Fixed exception loss on `withContext` cancellation (#675). + +## Version 0.30.1 + +Maintenance release: +* Added `Dispatchers.Main` to common dispatchers, which can be used from Android, Swing and JavaFx projects if a corresponding integration library is added to dependencies. +* With `Dispatchers.Main` improvement tooling bug in Android Studio #626 is mitigated, so Android users now can safely start the migration to the latest `kotlinx.coroutines` version. +* Fixed bug with thread unsafety of shutdown sequence in `EventLoop`. +* Experimental coroutine dispatcher now has `close` contract similar to Java `Executor`, so it can be safely instantiated and closed multiple times (affects only unit tests). +* Atomicfu version is updated with fixes in JS transformer (see #609) + +## Version 0.30.0 + +* **[Major]** Further improvements in exception handling — no failure exception is lost. + * `async` and async-like builders cancel parent on failure (it affects `CompletableDeferred`, and all reactive integration builders). + * This makes parallel decomposition exception-safe and reliable without having to rember about `awaitAll` (see #552). + * `Job()` wih parent now also cancels parent on failure consistently with other scopes. + * All coroutine builders and `Job` implementations propagate failure to the parent unless it is a `CancellationException`. + * Note, "scoping" builders don't "cancel the parent" verbatim, but rethrow the corresponding exception to the caller for handling. + * `SupervisorJob()` and `supervisorScope { ... }` are introduced, allowing for a flexible implementation of custom exception-handling policies, see a [new section in the guide on supervision](docs/topics/exception-handling.md#supervision). + * Got rid of `awaitAll` in documentation and rewrote `currentScope` section (see #624). +* **[Major]** Coroutine scheduler is used for `Dispatchers.Default` by default instead of deprecated `CommonPool`. + * "`DefaultDispatcher`" is used as a public name of the default impl (you'll see it thread names and in the guide). + * `-Dkotlinx.coroutines.scheduler=off` can be used to switch back to `CommonPool` for a time being (until deprecated CommonPool is removed). +* Make `CoroutineStart.ATOMIC` experimental as it covers important use-case with resource cleanup in finally block (see #627). +* Restored binary compatibility of `Executor.asCoroutineDispatcher` (see #629). +* Fixed OOM in thread-pool dispatchers (see #571). +* Check for cancellation when starting coroutine with `Dispatchers.Unconfined` (see #621). +* A bunch of various performance optimizations and docs fixes, including contributions from @AlexanderPrendota, @PaulWoitaschek. + +## Version 0.27.0 + +* **[Major]** Public API revision. All public API was reviewed and marked as preparation to `1.0` release: + 1. `@Deprecated` API. All API marked as deprecated will be removed in 1.0 release without replacement. + 2. `@ExperimentalCoroutinesApi` API. This API is experimental and may change in the future, but migration mechanisms will be provided. Signature, binary compatibility and semantics can be changed. + 3. `@InternalCoroutinesApi`. This API is intended to be used **only** from within `kotlinx.coroutines`. It can and will be changed, broken + and removed in the future releases without any warnings and migration aids. If you find yourself using this API, it is better to report + your use-case to Github issues, so decent, stable and well-tested alternative can be provided. + 4. `@ObsoleteCoroutinesApi`. This API has serious known flaws and will be replaced with a better alternative in the nearest releases. + 5. Regular public API. This API is proven to be stable and is not going to be changed. If at some point it will be discovered that such API + has unfixable design flaws, it will be gradually deprecated with proper replacement and migration aid, but won't be removed for at least a year. +* **[Major]** Job state machine is reworked. It includes various performance improvements, fixes in +data-races which could appear in a rare circumstances and consolidation of cancellation and exception handling. +Visible consequences of include more robust exception handling for large coroutines hierarchies and for different kinds of `CancellationException`, transparent parallel decomposition and consistent view of coroutines hierarchy in terms of its state (see #220 and #585). +* NIO, Quasar and Rx1 integration modules are removed with no replacement (see #595, #601, #603). +* `withContext` is now aligned with structured concurrency and awaits for all launched tasks, its performance is significantly improved (see #553 and #617). +* Added integration module with Play Services Task API. Thanks @SUPERCILEX and @lucasvalenteds for the contribution! +* Integration with Rx2 now respects nullability in type constraints (see #347). Thanks @Dmitry-Borodin for the contribution! +* `CompletableFuture.await` and `ListenableFuture.await` now propagate cancellation to the future (see #611). +* Cancellation of `runBlocking` machinery is improved (see #589). +* Coroutine guide is restructured and split to multiple files for the sake of simplicity. +* `CoroutineScope` factory methods add `Job` if it is missing from the context to enforce structured concurrency (see #610). +* `Handler.asCoroutineDispatcher` has a `name` parameter for better debugging (see #615). +* Fixed bug when `CoroutineSchedule` was closed from one of its threads (see #612). +* Exceptions from `CoroutineExceptionHandler` are reported by default exception handler (see #562). +* `CoroutineName` is now available from common modules (see #570). +* Update to Kotlin 1.2.70. + +## Version 0.26.1 + +* Android `Main` dispatcher is `async` by default which may significantly improve UI performance. Contributed by @JakeWharton (see #427). +* Fixed bug when lazily-started coroutine with registered cancellation handler was concurrently started and cancelled. +* Improved termination sequence in IO dispatcher. +* Fixed bug with `CoroutineScope.plus` operator (see #559). +* Various fixes in the documentation. Thanks to @SUPERCILEX, @yorlov, @dualscyther and @soudmaijer! + +## Version 0.26.0 + +* Major rework of `kotlinx.coroutines` concurrency model (see #410 for a full explanation of the rationale behind this change): + * All coroutine builders are now extensions on `CoroutineScope` and inherit its `coroutineContext`. Standalone builders are deprecated. + * As a consequence, all nested coroutines launched via builders now automatically establish parent-child relationship and inherit `CoroutineDispatcher`. + * All coroutine builders use `Dispatchers.Default` by default if `CoroutineInterceptor` is not present in their context. + * [CoroutineScope](https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/-coroutine-scope/index.html) became the first-class citizen in `kolinx.coroutines`. + * `withContext` `block` argument has `CoroutineScope` as a receiver. + * [GlobalScope](https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/-global-scope/index.html) is introduced to simplify migration to new API and to launch global-level coroutines. + * `currentScope` and `coroutineScope` builders are introduced to extract and provide `CoroutineScope`. + * Factory methods to create `CoroutineScope` from `CoroutineContext` are introduced. + * `CoroutineScope.isActive` became an extension property. + * New sections about structured concurrency in core guide: ["Structured concurrency"](docs/topics/coroutines-guide.md#structured-concurrency), ["Scope builder"](docs/topics/coroutines-guide.md#scope-builder) and ["Structured concurrency with async"](docs/topics/coroutines-guide.md#structured-concurrency-with-async). + * New section in UI guide with Android example: ["Structured concurrency, lifecycle and coroutine parent-child hierarchy"](ui/coroutines-guide-ui.md#structured-concurrency,-lifecycle-and-coroutine-parent-child-hierarchy). + * Deprecated reactive API is removed. +* Dispatchers are renamed and grouped in the Dispatchers object (see #41 and #533): + * Dispatcher names are consistent. + * Old dispatchers including `CommonPool` are deprecated. +* Fixed bug with JS error in rare cases in `invokeOnCompletion(onCancelling = true)`. +* Fixed loading of Android exception handler when `Thread.contextClassLoader` is mocked (see #530). +* Fixed bug when `IO` dispatcher silently hung (see #524 and #525) . + +## Version 0.25.3 + +* Distribution no longer uses multi-version jar which is not supported on Android (see #510). +* JS version of the library does not depend on AtomicFu anymore: + All the atomic boxes in JS are fully erased. +* Note that versions 0.25.1-2 are skipped for technical reasons (they were not fully released). + +## Version 0.25.0 + +* Major rework on exception-handling and cancellation in coroutines (see #333, #452 and #451): + * New ["Exception Handling" section in the guide](docs/topics/coroutines-guide.md#exception-handling) explains exceptions in coroutines. + * Semantics of `Job.cancel` resulting `Boolean` value changed — `true` means exception was handled by the job, caller shall handle otherwise. + * Exceptions are properly propagated from children to parents. + * Installed `CoroutineExceptionHandler` for a family of coroutines receives one aggregated exception in case of failure. + * Change `handleCoroutineException` contract, so custom exception handlers can't break coroutines machinery. + * Unwrap `JobCancellationException` properly to provide exception transparency over whole call chain. +* Introduced support for thread-local elements in coroutines context (see #119): + * `ThreadContextElement` API for custom thread-context sensitive context elements. + * `ThreadLocal.asContextElement()` extension function to convert an arbitrary thread-local into coroutine context element. + * New ["Thread-local data" subsection in the guide](docs/topics/coroutines-guide.md#thread-local-data) with examples. + * SLF4J Mapped Diagnostic Context (MDC) integration is provided via `MDCContext` element defined in [`kotlinx-coroutines-slf4j`](integration/kotlinx-coroutines-slf4j/README.md) integration module. +* Introduced IO dispatcher to offload blocking I/O-intensive tasks (see #79). +* Introduced `ExecutorCoroutineDispatcher` instead of `CloseableCoroutineDispatcher` (see #385). +* Built with Kotlin 1.2.61 and Kotlin/Native 0.8.2. +* JAR files for `kotlinx-coroutines` are now [JEP 238](https://openjdk.java.net/jeps/238) multi-release JAR files. + * On JDK9+ `VarHandle` is used for atomic operations instead of `Atomic*FieldUpdater` for better performance. + * See [AtomicFu](https://github.com/Kotlin/kotlinx.atomicfu/blob/master/README.md) project for details. +* Reversed addition of `BlockingChecker` extension point to control where `runBlocking` can be used (see #227). + * `runBlocking` can be used anywhere without limitations (again), but it would still cause problems if improperly used on UI thread. +* Corrected return-type of `EventLoop` pseudo-constructor (see #477, PR by @Groostav). +* Fixed `as*Future()` integration functions to catch all `Throwable` exceptions (see #469). +* Fixed `runBlocking` cancellation (see #501). +* Fixed races and timing bugs in `withTimeoutOrNull` (see #498). +* Execute `EventLoop.invokeOnTimeout` in `DefaultDispatcher` to allow busy-wait loops inside `runBlocking` (see #479). +* Removed `kotlinx-coroutines-io` module from the project, it has moved to [kotlinx-io](https://github.com/kotlin/kotlinx-io/). +* Provide experimental API to create limited view of experimental dispatcher (see #475). +* Various minor fixes by @LouisCAD, @Dmitry-Borodin. + +## Version 0.24.0 + +* Fully multiplatform release with Kotlin/Native support (see #246): + * Only single-threaded operation inside `runBlocking` event loop is supported at this moment. + * See details on setting up build environment [here](native/README.md). +* Improved channels: + * Introduced `SendChannel.invokeOnClose` (see #341). + * Make `close`, `cancel`, `isClosedForSend`, `isClosedForReceive` and `offer` linearizable with other operations (see #359). + * Fixed bug when send operation can be stuck in channel forever. + * Fixed broadcast channels on JS (see #412). +* Provides `BlockingChecker` mechanism which checks current context (see #227). + * Attempts to use `runBlocking` from any supported UI thread (Android, JavaFx, Swing) will result in exception. +* Android: + * Worked around Android bugs with zero-size ForkJoinPool initialization (see #432, #288). + * Introduced `UI.immediate` extension as performance-optimization to immediately execute tasks which are invoked from the UI thread (see #381). + * Use it only when absolutely needed. It breaks asynchrony of coroutines and may lead to surprising and unexpected results. +* Fixed materialization of a `cause` exception for `Job` onCancelling handlers (see #436). +* Fixed JavaFx `UI` on Java 9 (see #443). +* Fixed and documented the order between cancellation handlers and continuation resume (see #415). +* Fixed resumption of cancelled continuation (see #450). +* Includes multiple fixes to documentation contributed by @paolop, @SahilLone, @rocketraman, @bdavisx, @mtopolnik, @Groostav. +* Experimental coroutines scheduler preview (JVM only): + * Written from scratch and optimized for communicating coroutines. + * Performs significantly better than ForkJoinPool on coroutine benchmarks and for connected applications with [ktor](https://ktor.io). + * Supports automatic creating of new threads for blocking operations running on the same thread pool (with an eye on solving #79), but there is no stable public API for it just yet. + * For preview, run JVM with `-Dkotlinx.coroutines.scheduler` option. In this case `DefaultDispatcher` is set to new experimental scheduler instead of FJP-based `CommonPool`. + * Submit your feedback to issue #261. + +## Version 0.23.4 + +* Recompiled with Kotlin 1.2.51 to solve broken metadata problem (see [KT-24944](https://youtrack.jetbrains.com/issue/KT-24944)). + +## Version 0.23.3 + +* Kotlin 1.2.50. +* JS: Moved to atomicfu version 0.10.3 that properly matches NPM & Kotlin/JS module names (see #396). +* Improve source-code compatibility with previous (0.22.x) version of `openChannel().use { ... }` pattern by providing deprecated extension function `use` on `ReceiveChannel`. + +## Version 0.23.2 + +* IO: fix joining and continuous writing byte array interference. + +## Version 0.23.1 + +* JS: Fix dependencies in NPM: add "kotlinx-atomicfu" dependency (see #370). +* Introduce `broadcast` coroutine builder (see #280): + * Support `BroadcastChannel.cancel` method to drop the buffer. + * Introduce `ReceiveChannel.broadcast()` extension. +* Fixed a bunch of doc typos (PRs by @paolop). +* Corrected previous version's release notes (PR by @ansman). + +## Version 0.23.0 + +* Kotlin 1.2.41 +* **Coroutines core module is made mostly cross-platform for JVM and JS**: + * Migrate channels and related operators to common, so channels can be used from JS (see #201). + * Most of the code is shared between JVM and JS versions using cross-platform version of [AtomicFU](https://github.com/Kotlin/kotlinx.atomicfu) library. + * The recent version of Kotlin allows default parameters in common code (see #348). + * The project is built using Gradle 4.6. +* **Breaking change**: `CancellableContinuation` is not a `Job` anymore (see #219): + * It does not affect casual users of `suspendCancellableCoroutine`, since all the typically used functions are still there. + * `CancellableContinuation.invokeOnCompletion` is deprecated now and its semantics had subtly changed: + * `invokeOnCancellation` is a replacement for `invokeOnCompletion` to install a handler. + * The handler is **not** invoked on `resume` which corresponds to the typical usage pattern. + * There is no need to check for `cont.isCancelled` in a typical handler code anymore (since handler is invoked only when continuation is cancelled). + * Multiple cancellation handlers cannot be installed. + * Cancellation handlers cannot be removed (disposed of) anymore. + * This change is designed to allow better performance of suspending cancellable functions: + * Now `CancellableContinuation` implementation has simpler state machine and is implemented more efficiently. + * Exception handling in `AbstractContinuation` (that implements `CancellableContinuation`) is now consistent: + * Always prefer exception thrown from coroutine as exceptional reason, add cancellation cause as suppressed exception. +* **Big change**: Deprecate `CoroutineScope.coroutineContext`: + * It is replaced with top-level `coroutineContext` function from Kotlin standard library. +* Improve `ReceiveChannel` operators implementations to guarantee closing of the source channels under all circumstances (see #279): + * `onCompletion` parameter added to `produce` and all other coroutine builders. + * Introduce `ReceiveChannel.consumes(): CompletionHandler` extension function. +* Replace `SubscriptionReceiveChannel` with `ReceiveChannel` (see #283, PR by @deva666). + * `ReceiveChannel.use` extension is introduced to preserve source compatibility, but is deprecated. + * `consume` or `consumeEach` extensions should be used for channels. + * When writing operators, `produce(onCompletion=consumes()) { ... }` pattern shall be used (see #279 above). +* JS: Kotlin is declared as peer dependency (see #339, #340, PR by @ansman). +* Invoke exception handler for actor on cancellation even when channel was successfully closed, so exceptions thrown by actor are always reported (see #368). +* Introduce `awaitAll` and `joinAll` for `Deferred` and `Job` lists correspondingly (see #171). +* Unwrap `CompletionException` exception in `CompletionStage.await` slow-path to provide consistent results (see #375). +* Add extension to `ExecutorService` to return `CloseableCoroutineDispatcher` (see #278, PR by @deva666). +* Fail with proper message during build if JDK_16 is not set (see #291, PR by @venkatperi). +* Allow negative timeouts in `delay`, `withTimeout` and `onTimeout` (see #310). +* Fix a few bugs (leaks on cancellation) in `delay`: + * Invoke `clearTimeout` on cancellation in JSDispatcher. + * Remove delayed task on cancellation from internal data structure on JVM. +* Introduce `ticker` function to create "ticker channels" (see #327): + * It provides analogue of RX `Observable.timer` for coroutine channels. + * It is currently supported on JVM only. +* Add a test-helper class `TestCoroutineContext` (see #297, PR by @streetsofboston). + * It is currently supported on JVM only. + * Ticker channels (#327) are not yet compatible with it. +* Implement a better way to set `CoroutineContext.DEBUG` value (see #316, PR by @dmytrodanylyk): + * Made `CoroutineContext.DEBUG_PROPERTY_NAME` constant public. + * Introduce public constants with `"on"`, `"off"`, `"auto"` values. +* Introduce system property to control `CommonPool` parallelism (see #343): + * `CommonPool.DEFAULT_PARALLELISM_PROPERTY_NAME` constant is introduced with a value of "kotlinx.coroutines.default.parallelism". +* Include package-list files into documentation site (see #290). +* Fix various typos in docs (PRs by @paolop and @ArtsiomCh). + +## Version 0.22.5 + +* JS: Fixed main file reference in [NPM package](https://www.npmjs.com/package/kotlinx-coroutines-core) +* Added context argument to `Channel.filterNot` (PR by @jcornaz). +* Implemented debug `toString` for channels (see #185). + +## Version 0.22.4 + +* JS: Publish to NPM (see #229). +* JS: Use node-style dispatcher on ReactNative (see #236). +* [jdk8 integration](integration/kotlinx-coroutines-jdk8/README.md) improvements: + * Added conversion from `CompletionStage` to `Deferred` (see #262, PR by @jcornaz). + * Use fast path in `CompletionStage.await` and make it cancellable. + +## Version 0.22.3 + +* Fixed `produce` builder to close the channel on completion instead of cancelling it, which lead to lost elements with buffered channels (see #256). +* Don't use `ForkJoinPool` if there is a `SecurityManager` present to work around JNLP problems (see #216, PR by @NikolayMetchev). +* JS: Check for undefined `window.addEventListener` when choosing default coroutine dispatcher (see #230, PR by @ScottPierce). +* Update 3rd party dependencies: + * [kotlinx-coroutines-rx1](reactive/kotlinx-coroutines-rx1) to RxJava version `1.3.6`. + * [kotlinx-coroutines-rx2](reactive/kotlinx-coroutines-rx2) to RxJava version `2.1.9`. + * [kotlinx-coroutines-guava](integration/kotlinx-coroutines-guava) to Guava version `24.0-jre`. + +## Version 0.22.2 + +* Android: Use @Keep annotation on AndroidExceptionPreHandler to fix the problem on Android with minification enabled (see #214). +* Reactive: Added `awaitFirstOrDefault` and `awaitFirstOrNull` extensions (see #224, PR by @konrad-kaminski). +* Core: Fixed `withTimeout` and `withTimeoutOrNull` that should not use equals on result (see #212, PR by @konrad-kaminski). +* Core: Fixed hanged receive from a closed subscription of BroadcastChannel (see #226). +* IO: fixed error propagation (see https://github.com/ktorio/ktor/issues/301). +* Include common sources into sources jar file to work around KT-20971. +* Fixed bugs in documentation due to MPP. + +## Version 0.22.1 + +* Migrated to Kotlin 1.2.21. +* Improved `actor` builder documentation (see #210) and fixed bugs in rendered documentation due to multiplatform. +* Fixed `runBlocking` to properly support specified dispatchers (see #209). +* Fixed data race in `Job` implementation (it was hanging at `LockFreeLinkedList.helpDelete` on certain stress tests). +* `AbstractCoroutine.onCancellation` is invoked before cancellation handler that is set via `invokeOnCompletion`. +* Ensure that `launch` handles uncaught exception before another coroutine that uses `join` on it resumes (see #208). + +## Version 0.22 + +* Migrated to Kotlin 1.2.20. +* Introduced stable public API for `AbstractCoroutine`: + * Implements `Job`, `Continuation`, and `CoroutineScope`. + * Has overridable `onStart`, `onCancellation`, `onCompleted` and `onCompletedExceptionally` functions. + * Reactive integration modules are now implemented using public API only. + * Notifies onXXX before all the installed handlers, so `launch` handles uncaught exceptions before "joining" coroutines wakeup (see #208). + +## Version 0.21.2 + +* Fixed `openSubscription` extension for reactive `Publisher`/`Observable`/`Flowable` when used with `select { ... }` and added an optional `request` parameter to specify how many elements are requested from publisher in advance on subscription (see #197). +* Simplified implementation of `Channel.flatMap` using `toChannel` function to work around Android 5.0 APK install SIGSEGV (see #205). + +## Version 0.21.1 + +* Improved performance of coroutine dispatching (`DispatchTask` instance is no longer allocated). +* Fixed `Job.cancel` and `CompletableDeferred.complete` to support cancelling/completing states and properly wait for their children to complete on join/await (see #199). +* Fixed a bug in binary heap implementation (used internally by `delay`) which could have resulted in wrong delay time in rare circumstances. +* Coroutines library for [Kotlin/JS](js/README.md): + * `Promise.asDeferred` immediately installs handlers to avoid "Unhandled promise rejection" warning. + * Use `window.postMessage` instead of `setTimeout` for coroutines inside the browser to avoid timeout throttling (see #194). + * Use custom queue in `Window.awaitAnimationFrame` to align all animations and reduce overhead. + * Introduced `Window.asCoroutineDispatcher()` extension function. + +## Version 0.21 + +* Migrated to Kotlin 1.2.10. +* Coroutines library for [Kotlin/JS](js/README.md) and [multiplatform projects](https://kotlinlang.org/docs/reference/multiplatform.html) (see #33): + * `launch` and `async` coroutine builders. + * `Job` and `Deferred` light-weight future with cancellation support. + * `delay` and `yield` top-level suspending functions. + * `await` extension for JS `Promise` and `asPromise`/`asDeferred` conversions. + * `promise` coroutine builder. + * `Job()` and `CompletableDeferred()` factories. + * Full support for parent-child coroutine hierarchies. + * `Window.awaitAnimationFrame` extension function. + * [Sample frontend Kotlin/JS application](js/example-frontend-js/README.md) with coroutine-driven animations. +* `run` is deprecated and renamed to `withContext` (see #134). +* `runBlocking` and `EventLoop` implementations optimized (see #190). + +## Version 0.20 + +* Migrated to Kotlin 1.2.0. +* Channels: + * Sequence-like `filter`, `map`, etc extensions on `ReceiveChannel` are introduced (see #88 by @fvasco and #69 by @konrad-kaminski). + * Introduced `ReceiveChannel.cancel` method. + * All operators on `ReceiveChannel` fully consume the original channel (`cancel` it when they are done) using a helper `consume` extension. + * Deprecated `ActorJob` and `ProducerJob`; `actor` now returns `SendChannel` and `produce` returns `ReceiveChannel` (see #127). + * `SendChannel.sendBlocking` extension method (see #157 by @@fvasco). +* Parent-child relations between coroutines: + * Introduced an optional `parent` job parameter for all coroutine builders so that code with an explict parent `Job` is more natural. + * Added `parent` parameter to `CompletableDeferred` constructor. + * Introduced `Job.children` property. + * `Job.cancelChildren` is now an extension (member is deprecated and hidden). + * `Job.joinChildren` extension is introduced. + * Deprecated `Job.attachChild` as a error-prone API. + * Fixed StackOverflow when waiting for a lot of completed children that did not remove their handlers from the parent. +* Use `java.util.ServiceLoader` to find default instances of `CoroutineExceptionHandler`. +* Android UI integration: + * Use `Thread.getUncaughtExceptionPreHandler` to make sure that exceptions are logged before crash (see #148). + * Introduce `UI.awaitFrame` for animation; added sample coroutine-based animation application for Android [here](ui/kotlinx-coroutines-android/animation-app). + * Fixed `delay(Long.MAX_VALUE)` (see #161) +* Added missing `DefaultDispatcher` on some reactive operators (see #174 by @fvasco) +* Fixed `actor` and `produce` so that a cancellation of a Job cancels the underlying channel (closes and removes all the pending messages). +* Fixed sporadic failure of `example-context-06` (see #160) +* Fixed hang of `Job.start` on lazy coroutine with attached `invokeOnCompletion` handler. +* A more gradual introduction to `runBlocking` and coroutines in the [guide](docs/topics/coroutines-guide.md) (see #166). + +## Version 0.19.3 + +* Fixed `send`/`openSubscription` race in `ArrayBroadcastChannel`. + This race lead to stalled (hanged) `send`/`receive` invocations. +* Project build has been migrated to Gradle. + +## Version 0.19.2 + +* Fixed `ArrayBroadcastChannel` receive of stale elements on `openSubscription`. + Only elements that are sent after invocation of `openSubscription` are received now. +* Added a default value for `context` parameter to `rxFlowable` (see #146 by @PhilGlass). +* Exception propagation logic from cancelled coroutines is adjusted (see #152): + * When cancelled coroutine crashes due to some other exception, this other exception becomes the cancellation reason + of the coroutine, while the original cancellation reason is suppressed. + * `UnexpectedCoroutineException` is no longer used to report those cases as is removed. + * This fixes a race between crash of CPU-consuming coroutine and cancellation which resulted in an unhandled exception + and lead to crashes on Android. +* `run` uses cancelling state & propagates exceptions when cancelled (see #147): + * When coroutine that was switched into a different dispatcher using `run` is cancelled, the run invocation does not + complete immediately, but waits until the body completes. + * If the body completes with exception, then this exception is propagated. +* No `Job` in `newSingleThreadContext` and `newFixedThreadPoolContext` anymore (see #149, #151): + * This resolves the common issue of using `run(ctx)` where ctx comes from either `newSingleThreadContext` or + `newFixedThreadPoolContext` invocation. They both used to return a combination of dispatcher + job, + and this job was overriding the parent job, thus preventing propagation of cancellation. Not anymore. + * `ThreadPoolDispatcher` class is now public and is the result type for both functions. + It has the `close` method to release the thread pool. + +## Version 0.19.1 + +* Failed parent Job cancels all children jobs, then waits for them them. + This makes parent-child hierarchies easier to get working right without + having to use `try/catch` or other exception handlers. +* Fixed a race in `ArrayBroadcastChannel` between `send` and `openChannel` invocations + (see #138). +* Fixed quite a rare race in `runBlocking` that resulted in `AssertionError`. + Unfortunately, cannot write a reliable stress-test to reproduce it. +* Updated Reactor support to leverage Bismuth release train + (contributed by @sdeleuze, see PR #141) + +## Version 0.19 + +* This release is published to Maven Central. +* `DefaultDispatcher` is introduced (see #136): + * `launch`, `async`, `produce`, `actor` and other integration-specific coroutine builders now use + `DefaultDispatcher` as the default value for their `context` parameter. + * When a context is explicitly specified, `newCoroutineContext` function checks if there is any + interceptor/dispatcher defined in the context and uses `DefaultDispatcher` if there is none. + * `DefaultDispatcher` is currently defined to be equal to `CommonPool`. + * Examples in the [guide](docs/topics/coroutines-guide.md) now start with `launch { ... }` code and explanation on the nature + and the need for coroutine context starts in "Coroutine context and dispatchers" section. +* Parent coroutines now wait for their children (see #125): + * Job _completing_ state is introduced in documentation as a state in which parent coroutine waits for its children. + * `Job.attachChild` and `Job.cancelChildren` are introduced. + * `Job.join` now always checks cancellation status of invoker coroutine for predictable behavior when joining + failed child coroutine. + * `Job.cancelAndJoin` extension is introduced. + * `CoroutineContext.cancel` and `CoroutineContext.cancelChildren` extensions are introduced for convenience. + * `withTimeout`/`withTimeoutOrNull` blocks become proper coroutines that have `CoroutineScope` and wait for children, too. + * Diagnostics in cancellation and unexpected exception messages are improved, + coroutine name is included in debug mode. + * Fixed cancellable suspending functions to throw `CancellationException` (as was documented before) even when + the coroutine is cancelled with another application-specific exception. + * `JobCancellationException` is introduced as a specific subclass of `CancellationException` which is + used for coroutines that are cancelled without cause and to wrap application-specific exceptions. + * `Job.getCompletionException` is renamed to `Job.getCancellationException` and return a wrapper exception if needed. + * Introduced `Deferred.getCompletionExceptionOrNull` to get not-wrapped exception result of `async` task. + * Updated docs for `Job` & `Deferred` to explain parent/child relations. +* `select` expression is modularized: + * `SelectClause(0,1,2)` interfaces are introduced, so that synchronization + constructs can define their select clauses without having to modify + the source of the `SelectBuilder` in `kotlinx-corounes-core` module. + * `Job.onJoin`, `Deferred.onAwait`, `Mutex.onLock`, `SendChannel.onSend`, `ReceiveChannel.onReceive`, etc + that were functions before are now properties returning the corresponding select clauses. Old functions + are left in bytecode for backwards compatibility on use-site, but any outside code that was implementing those + interfaces by itself must be updated. + * This opens road to moving channels into a separate module in future updates. +* Renamed `TimeoutException` to `TimeoutCancellationException` (old name is deprecated). +* Fixed various minor problems: + * JavaFx toolkit is now initialized by `JavaFx` context (see #108). + * Fixed lost ACC_STATIC on <clinit> methods (see #116). + * Fixed link to source code from documentation (see #129). + * Fixed `delay` in arbitrary contexts (see #133). +* `kotlinx-coroutines-io` module is introduced. It is a work-in-progress on `ByteReadChannel` and `ByteWriteChannel` + interfaces, their implementations, and related classes to enable convenient coroutine integration with various + asynchronous I/O libraries and sockets. It is currently _unstable_ and **will change** in the next release. + +## Version 0.18 + +* Kotlin 1.1.4 is required to use this version, which enables: + * `withLock` and `consumeEach` functions are now inline suspend functions. + * `JobSupport` class implementation is optimized (one fewer field). +* `TimeoutException` is public (see #89). +* Improvements to `Mutex` (courtesy of @fvasco): + * Introduced `holdsLock` (see #92). + * Improved documentation on `Mutex` fairness (see #90). +* Fixed NPE when `ArrayBroadcastChannel` is closed concurrently with receive (see #97). +* Fixed bug in internal class LockFreeLinkedList that resulted in ISE under stress in extremely rare circumstances. +* Integrations: + * [quasar](integration/kotlinx-coroutines-quasar): Introduced integration with suspendable JVM functions + that are instrumented with [Parallel Universe Quasar](https://docs.paralleluniverse.co/quasar/) + (thanks to the help of @pron). + * [reactor](reactive/kotlinx-coroutines-reactor): Replaced deprecated `setCancellation` with `onDipose` and + updated to Aluminium-SR3 release (courtesy of @yxf07, see #96) + * [jdk8](integration/kotlinx-coroutines-jdk8): Added adapters for `java.time` classes (courtesy of @fvasco, see #93) + +## Version 0.17 + +* `CompletableDeferred` is introduced as a set-once event-like communication primitive (see #70). + * [Coroutines guide](docs/topics/coroutines-guide.md) uses it in a section on actors. + * `CompletableDeferred` is an interface with private impl (courtesy of @fvasco, see #86). + * It extends `Deferred` interface with `complete` and `completeExceptionally` functions. +* `Job.join` and `Deferred.await` wait until a cancelled coroutine stops execution (see #64). + * `Job` and `Deferred` have a new _cancelling_ state which they enter on invocation of `cancel`. + * `Job.invokeOnCompletion` has an additional overload with `onCancelling: Boolean` parameter to + install handlers that are fired as soon as coroutine enters _cancelling_ state as opposed + to waiting until it _completes_. + * Internal `select` implementation is refactored to decouple it from `JobSupport` internal class + and to optimize its state-machine. + * Internal `AbstractCoroutine` class is refactored so that it is extended only by true coroutines, + all of which support the new _cancelling_ state. +* `CoroutineScope.context` is renamed to `coroutineContext` to avoid conflicts with other usages of `context` + in applications (like Android context, see #75). +* `BroadcastChannel.open` is renamed to `openSubscription` (see #54). +* Fixed `StackOverflowError` in a convoy of `Mutex.unlock` invokers with `Unconfined` dispatcher (see #80). +* Fixed `SecurityException` when trying to use coroutines library with installed `SecurityManager`. +* Fixed a bug in `withTimeoutOrNull` in case with nested timeouts when coroutine was cancelled before it was + ever suspended. +* Fixed a minor problem with `awaitFirst` on reactive streams that would have resulted in spurious stack-traces printed + on the console when used with publishers/observables that continue to invoke `onNext` despite being cancelled/disposed + (which they are technically allowed to do by specification). +* All factory functions for various interfaces are implemented as top-level functions + (affects `Job`, `Channel`, `BroadcastChannel`, `Mutex`, `EventLoop`, and `CoroutineExceptionHandler`). + Previous approach of using `operator invoke` on their companion objects is deprecated. +* Nicer-to-use debug `toString` implementations for coroutine dispatcher tasks and continuations. +* A default dispatcher for `delay` is rewritten and now shares code with `EventLoopImpl` that is used by + `runBlocking`. It internally supports non-default `TimeSource` so that delay-using tests can be written + with "virtual time" by replacing their time source for the duration of tests (this feature is not available + outside of the library). + +## Version 0.16 + +* Coroutines that are scheduled for execution are cancellable by default now + * `suspendAtomicCancellableCoroutine` function is introduced for funs like + `send`/`receive`/`receiveOrNull` that require atomic cancellation + (they cannot be cancelled after decision was made) + * Coroutines started with default mode using + `async`/`launch`/`actor` builders can be cancelled before their execution starts + * `CoroutineStart.ATOMIC` is introduced as a start mode to specify that + coroutine cannot be cancelled before its execution starts + * `run` function is also cancellable in the same way and accepts an optional + `CoroutineStart` parameter to change this default. +* `BroadcastChannel` factory function is introduced +* `CoroutineExceptionHandler` factory function is introduced by @konrad-kaminski +* [`integration`](integration) directory is introduced for all 3rd party integration projects + * It has [contribution guidelines](integration/README.md#contributing) and contributions from community are welcome + * Support for Guava `ListenableFuture` in the new [`kotlinx-coroutines-guava`](integration/kotlinx-coroutines-guava) module + * Rx1 Scheduler support by @konrad-kaminski +* Fixed a number of `Channel` and `BroadcastChannel` implementation bugs related to concurrent + send/close/close of channels that lead to hanging send, offer or close operations (see #66). + Thanks to @chrisly42 and @cy6erGn0m for finding them. +* Fixed `withTimeoutOrNull` which was returning `null` on timeout of inner or outer `withTimeout` blocks (see #67). + Thanks to @gregschlom for finding the problem. +* Fixed a bug where `Job` fails to dispose a handler when it is the only handler by @uchuhimo + +## Version 0.15 + +* Switched to Kotlin version 1.1.2 (can still be used with 1.1.0). +* `CoroutineStart` enum is introduced for `launch`/`async`/`actor` builders: + * The usage of `luanch(context, start = false)` is deprecated and is replaced with + `launch(context, CoroutineStart.LAZY)` + * `CoroutineStart.UNDISPATCHED` is introduced to start coroutine execution immediately in the invoker thread, + so that `async(context, CoroutineStart.UNDISPATCHED)` is similar to the behavior of C# `async`. + * [Guide to UI programming with coroutines](ui/coroutines-guide-ui.md) mentions the use of it to optimize + the start of coroutines from UI threads. +* Introduced `BroadcastChannel` interface in `kotlinx-coroutines-core` module: + * It extends `SendChannel` interface and provides `open` function to create subscriptions. + * Subscriptions are represented with `SubscriptionReceiveChannel` interface. + * The corresponding `SubscriptionReceiveChannel` interfaces are removed from [reactive](reactive) implementation + modules. They use an interface defined in `kotlinx-coroutines-core` module. + * `ConflatedBroadcastChannel` implementation is provided for state-observation-like use-cases, where a coroutine or a + regular code (in UI, for example) updates the state that subscriber coroutines shall react to. + * `ArrayBroadcastChannel` implementation is provided for event-bus-like use-cases, where a sequence of events shall + be received by multiple subscribers without any omissions. + * [Guide to reactive streams with coroutines](reactive/coroutines-guide-reactive.md) includes + "Rx Subject vs BroadcastChannel" section. +* Pull requests from Konrad Kamiński are merged into reactive stream implementations: + * Support for Project Reactor `Mono` and `Flux`. + See [`kotlinx-coroutines-reactor`](reactive/kotlinx-coroutines-reactor) module. + * Implemented Rx1 `Completable.awaitCompleted`. + * Added support for Rx2 `Maybe`. +* Better timeout support: + * Introduced `withTimeoutOrNull` function. + * Implemented `onTimeout` clause for `select` expressions. + * Fixed spurious concurrency inside `withTimeout` blocks on their cancellation. + * Changed behavior of `withTimeout` when `CancellationException` is suppressed inside the block. + Invocation of `withTimeout` now always returns the result of execution of its inner block. +* The `channel` property in `ActorScope` is promoted to a wider `Channel` type, so that an actor + can have an easy access to its own inbox send channel. +* Renamed `Mutex.withMutex` to `Mutex.withLock`, old name is deprecated. + +## Version 0.14 + +* Switched to Kotlin version 1.1.1 (can still be used with 1.1.0). +* Introduced `consumeEach` helper function for channels and reactive streams, Rx 1.x, and Rx 2.x. + * It ensures that streams are unsubscribed from on any exception. + * Iteration with `for` loop on reactive streams is **deprecated**. + * [Guide to reactive streams with coroutines](reactive/coroutines-guide-reactive.md) is updated virtually + all over the place to reflect these important changes. +* Implemented `awaitFirstOrDefault` extension for reactive streams, Rx 1.x, and Rx 2.x. +* Added `Mutex.withMutex` helper function. +* `kotlinx-coroutines-android` module has `provided` dependency on of Android APIs to + eliminate warnings when using it in android project. + +## Version 0.13 + +* New `kotlinx-coroutinex-android` module with Android `UI` context implementation. +* Introduced `whileSelect` convenience function. +* Implemented `ConflatedChannel`. +* Renamed various `toXXX` conversion functions to `asXXX` (old names are deprecated). +* `run` is optimized with fast-path case and no longer has `CoroutineScope` in its block. +* Fixed dispatching logic of `withTimeout` (removed extra dispatch). +* `EventLoop` that is used by `runBlocking` now implements Delay, giving more predictable test behavior. +* Various refactorings related to resource management and timeouts: + * `Job.Registration` is renamed to `DisposableHandle`. + * `EmptyRegistration` is renamed to `NonDisposableHandle`. + * `Job.unregisterOnCompletion` is renamed to `Job.disposeOnCompletion`. + * `Delay.invokeOnTimeout` is introduced. + * `withTimeout` now uses `Delay.invokeOnTimeout` when available. +* A number of improvement for reactive streams and Rx: + * Introduced `rxFlowable` builder for Rx 2.x. + * `Scheduler.asCoroutineDispatcher` extension for Rx 2.x. + * Fixed bug with sometimes missing `onComplete` in `publish`, `rxObservable`, and `rxFlowable` builders. + * Channels that are open for reactive streams are now `Closeable`. + * Fixed `CompletableSource.await` and added test for it. + * Removed `rx.Completable.await` due to name conflict. +* New documentation: + * [Guide to UI programming with coroutines](ui/coroutines-guide-ui.md) + * [Guide to reactive streams with coroutines](reactive/coroutines-guide-reactive.md) +* Code is published to JCenter repository. + +## Version 0.12 + +* Switched to Kotlin version 1.1.0 release. +* Reworked and updated utilities for + [Reactive Streams](kotlinx-coroutines-reactive), + [Rx 1.x](kotlinx-coroutines-rx1), and + [Rx 2.x](kotlinx-coroutines-rx2) with library-specific + coroutine builders, suspending functions, converters and iteration support. +* `LinkedListChannel` with unlimited buffer (`offer` always succeeds). +* `onLock` select clause and an optional `owner` parameter in all `Mutex` functions. +* `selectUnbiased` function. +* `actor` coroutine builder. +* Couple more examples for "Shared mutable state and concurrency" section and + "Channels are fair" section with ping-pong table example + in [coroutines guide](docs/topics/coroutines-guide.md). + +## Version 0.11-rc + +* `select` expression with onJoin/onAwait/onSend/onReceive clauses. +* `Mutex` is moved to `kotlinx.coroutines.sync` package. +* `ClosedSendChannelException` is a subclass of `CancellationException` now. +* New sections on "Shared mutable state and concurrency" and "Select expression" + in [coroutines guide](docs/topics/coroutines-guide.md). + +## Version 0.10-rc + +* Switched to Kotlin version 1.1.0-rc-91. +* `Mutex` synchronization primitive is introduced. +* `buildChannel` is renamed to `produce`, old name is deprecated. +* `Job.onCompletion` is renamed to `Job.invokeOnCompletion`, old name is deprecated. +* `delay` implementation in Swing, JavaFx, and scheduled executors is fixed to avoid an extra dispatch. +* `CancellableContinuation.resumeUndispatched` is introduced to make this efficient implementation possible. +* Remove unnecessary creation of `CancellationException` to improve performance, plus other performance improvements. +* Suppress deprecated and internal APIs from docs. +* Better docs at top level with categorized summary of classes and functions. + +## Version 0.8-beta + +* `defer` coroutine builder is renamed to `async`. +* `lazyDefer` is deprecated, `async` has an optional `start` parameter instead. +* `LazyDeferred` interface is deprecated, lazy start functionality is integrated into `Job` interface. +* `launch` has an optional `start` parameter for lazily started coroutines. +* `Job.start` and `Job.isCompleted` are introduced. +* `Deferred.isCompletedExceptionally` and `Deferred.isCancelled` are introduced. +* `Job.getInactiveCancellationException` is renamed to `getCompletionException`. +* `Job.join` is now a member function. +* Internal `JobSupport` state machine is enhanced to support _new_ (not-started-yet) state. + So, lazy coroutines do not need a separate state variable to track their started/not-started (new/active) status. +* Exception transparency in `Job.cancel` (original cause is rethrown). +* Clarified possible states for `Job`/`CancellableContinuation`/`Deferred` in docs. +* Example on async-style functions and links to API reference site from [coroutines guide](docs/topics/coroutines-guide.md). + +## Version 0.7-beta + +* Buffered and unbuffered channels are introduced: `Channel`, `SendChannel`, and `ReceiveChannel` interfaces, + `RendezvousChannel` and `ArrayChannel` implementations, `Channel()` factory function and `buildChannel{}` + coroutines builder. +* `Here` context is renamed to `Unconfined` (the old name is deprecated). +* A [guide on coroutines](docs/topics/coroutines-guide.md) is expanded: sections on contexts and channels. + +## Version 0.6-beta + +* Switched to Kotlin version 1.1.0-beta-37. +* A [guide on coroutines](docs/topics/coroutines-guide.md) is expanded. + +## Version 0.5-beta + +* Switched to Kotlin version 1.1.0-beta-22 (republished version). +* Removed `currentCoroutineContext` and related thread-locals without replacement. + Explicitly pass coroutine context around if needed. +* `lazyDefer(context) {...}` coroutine builder and `LazyDeferred` interface are introduced. +* The default behaviour of all coroutine dispatchers is changed to always schedule execution of new coroutine + for later in this thread or thread pool. Correspondingly, `CoroutineDispatcher.isDispatchNeeded` function + has a default implementation that returns `true`. +* `NonCancellable` context is introduced. +* Performance optimizations for cancellable continuations (fewer objects created). +* A [guide on coroutines](docs/topics/coroutines-guide.md) is added. + +## Version 0.4-beta + +* Switched to Kotlin version 1.1.0-beta-18 (republished version). +* `CoroutineDispatcher` methods now have `context` parameter. +* Introduced `CancellableContinuation.isCancelled` +* Introduced `EventLoop` dispatcher and made it a default for `runBlocking { ... }` +* Introduced `CoroutineScope` interface with `isActive` and `context` properties; + standard coroutine builders include it as receiver for convenience. +* Introduced `Executor.toCoroutineDispatcher()` extension. +* Delay scheduler thread is not daemon anymore, but times out automatically. +* Debugging facilities in `newCoroutineContext` can be explicitly disabled with `-Dkotlinx.coroutines.debug=off`. +* xxx-test files are renamed to xxx-example for clarity. +* Fixed NPE in Job implementation when starting coroutine with already cancelled parent job. +* Support cancellation in `kotlinx-coroutines-nio` module diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 77b727b4..cc9c3bc9 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -7,21 +7,8 @@ fixes/changes/improvements via pull requests. Both bug reports and feature requests are welcome. Submit issues [here](https://github.com/Kotlin/kotlinx.coroutines/issues). - -* Search for existing issues to avoid reporting duplicates. -* When submitting a bug report: - * Test it against the most recently released version. It might have been already fixed. - * By default, we assume that your problem reproduces in Kotlin/JVM. Please, mention if the problem is - specific to Kotlin/JS or Kotlin/Native. - * Include the code that reproduces the problem. Provide the complete reproducer code, yet minimize it as much as possible. - * However, don't put off reporting any weird or rarely appearing issues just because you cannot consistently - reproduce them. - * If the bug is in behavior, then explain what behavior you've expected and what you've got. -* When submitting a feature request: - * Explain why you need the feature — what's your use-case, what's your domain. - * Explaining the problem you face is more important than suggesting a solution. - Report your problem even if you don't have any proposed solution. - * If there is an alternative way to do what you need, then show the code of the alternative. +Questions about usage and general inquiries are better suited for [StackOverflow](https://stackoverflow.com) +or the `#coroutines` channel in [KotlinLang Slack](https://surveys.jetbrains.com/s3/kotlin-slack-sign-up). ## Submitting PRs @@ -30,21 +17,23 @@ However, please keep in mind that maintainers will have to support the resulting so do familiarize yourself with the following guidelines. * All development (both new features and bug fixes) is performed in the `develop` branch. - * The `master` branch always contains sources of the most recently released version. - * Base PRs against the `develop` branch. + * The `master` branch contains the sources of the most recently released version. + * Base your PRs against the `develop` branch. * The `develop` branch is pushed to the `master` branch during release. * Documentation in markdown files can be updated directly in the `master` branch, unless the documentation is in the source code, and the patch changes line numbers. * If you fix documentation: * After fixing/changing code examples in the [`docs`](docs) folder or updating any references in the markdown files run the [Knit tool](#running-the-knit-tool) and commit the resulting changes as well. - It will not pass the tests otherwise. + The tests will not pass otherwise. * If you plan extensive rewrites/additions to the docs, then please [contact the maintainers](#contacting-maintainers) - to coordinate the work in advance. + to coordinate the work in advance. * If you make any code changes: * Follow the [Kotlin Coding Conventions](https://kotlinlang.org/docs/reference/coding-conventions.html). Use 4 spaces for indentation. - * [Build the project](#building) to make sure it all works and passes the tests. + Do not add extra newlines in function bodies: if you feel that blocks of code should be logically separated, + then separate them with a comment instead. + * [Build the project](#building) to make sure everything works and passes the tests. * If you fix a bug: * Write the test that reproduces the bug. * Fixes without tests are accepted only in exceptional circumstances if it can be shown that writing the @@ -53,37 +42,37 @@ so do familiarize yourself with the following guidelines. name test functions as `testXxx`. Don't use backticks in test names. * If you introduce any new public APIs: * Comment on the existing issue if you want to work on it or create one beforehand. - Ensure that the issue not only describes a problem, but also describes a solution that had received a positive feedback. Propose a solution if there isn't any. - PRs with new API, but without a corresponding issue with a positive feedback about the proposed implementation are unlikely to - be approved or reviewed. + Ensure that not only the issue describes a problem, but also that the proposed solution has received positive + feedback. Propose a solution if there isn't any. + PRs that add new API without a corresponding issue with positive feedback about the proposed implementation are + unlikely to be approved or reviewed. * All new APIs must come with documentation and tests. - * All new APIs are initially released with `@ExperimentalCoroutineApi` annotation and are graduated later. + * All new APIs are initially released with the `@ExperimentalCoroutineApi` annotation and graduate later. * [Update the public API dumps](#updating-the-public-api-dump) and commit the resulting changes as well. It will not pass the tests otherwise. - * If you plan large API additions, then please start by submitting an issue with the proposed API design + * If you plan large API additions, then please start by submitting an issue with the proposed API design to gather community feedback. - * [Contact the maintainers](#contacting-maintainers) to coordinate any big piece of work in advance. -* Steps for contributing new integration modules are explained [here](integration/README.md#Contributing). + * [Contact the maintainers](#contacting-maintainers) to coordinate any extensive work in advance. ## Building This library is built with Gradle. * Run `./gradlew build` to build. It also runs all the tests. -* Run `./gradlew <module>:test` to test the module you are looking at to speed +* Run `./gradlew <module>:check` to test the module you are looking at to speed things up during development. -* Run `./gradlew jvmTest` to perform only fast JVM tests of the core multiplatform module. +* Run `./gradlew <module>:jvmTest` to perform only the fast JVM tests of a multiplatform module. You can import this project into IDEA, but you have to delegate build actions -to Gradle (in Preferences -> Build, Execution, Deployment -> Build Tools -> Gradle -> Runner) +to Gradle (in Preferences -> Build, Execution, Deployment -> Build Tools -> Gradle -> Build and run). ### Environment requirements * JDK >= 11 referred to by the `JAVA_HOME` environment variable. * JDK 1.8 referred to by the `JDK_18` environment variable. Only used by nightly stress-tests. - It is OK to have `JDK_18` to a non 1.8 JDK (e.g. `JAVA_HOME`) for external contributions. + It is OK to have `JDK_18` point to a non-1.8 JDK (e.g. `JAVA_HOME`) for external contributions. -For external contributions you can for example add this to your shell startup scripts (e.g. `~/.zshrc`): +For external contributions you can, for example, add this to your shell startup scripts (e.g. `~/.zshrc`): ```shell # This assumes JAVA_HOME is set already to a JDK >= 11 version export JDK_18="$JAVA_HOME" @@ -92,18 +81,18 @@ export JDK_18="$JAVA_HOME" ### Running the Knit tool * Use [Knit](https://github.com/Kotlin/kotlinx-knit/blob/main/README.md) for updates to documentation: - * Run `./gradlew knit` to update example files, links, tables of content. - * Commit updated documents and examples together with other changes. + * Run `./gradlew knit` to update the example files, links, tables of content. + * Commit the updated documents and examples together with other changes. ### Updating the public API dump -* Use [Binary Compatibility Validator](https://github.com/Kotlin/binary-compatibility-validator/blob/master/README.md) for updates to public API: +* Use the [Binary Compatibility Validator](https://github.com/Kotlin/binary-compatibility-validator/blob/master/README.md) for updates to public API: * Run `./gradlew apiDump` to update API index files. - * Commit updated API indexes together with other changes. + * Commit the updated API indexes together with other changes. ## Releases -* Full release procedure checklist is [here](RELEASE.md). +* The full release procedure checklist is [here](RELEASE.md). ## Contacting maintainers @@ -1,3 +1,7 @@ +# This project was upgraded with external_updater. +# Usage: tools/external_updater/updater.sh update kotlinx.coroutines +# For more info, check https://cs.android.com/android/platform/superproject/+/master:tools/external_updater/README.md + name: "kotlinx.coroutines" description: "Library support for Kotlin coroutines" third_party { @@ -5,11 +9,11 @@ third_party { type: GIT value: "https://github.com/Kotlin/kotlinx.coroutines" } - version: "1.6.4" + version: "1.7.2" license_type: NOTICE last_upgrade_date { - year: 2022 + year: 2023 month: 7 - day: 20 + day: 12 } } @@ -3,12 +3,12 @@ [](https://kotlinlang.org/docs/components-stability.html) [](https://confluence.jetbrains.com/display/ALL/JetBrains+on+GitHub) [](https://www.apache.org/licenses/LICENSE-2.0) -[](https://search.maven.org/artifact/org.jetbrains.kotlinx/kotlinx-coroutines-core/1.6.4/pom) -[](http://kotlinlang.org) +[](https://central.sonatype.com/artifact/org.jetbrains.kotlinx/kotlinx-coroutines-core/1.7.2) +[](http://kotlinlang.org) [](https://kotlinlang.slack.com/messages/coroutines/) Library support for Kotlin coroutines with [multiplatform](#multiplatform) support. -This is a companion version for the Kotlin `1.6.21` release. +This is a companion version for the Kotlin `1.8.20` release. ```kotlin suspend fun main() = coroutineScope { @@ -20,7 +20,7 @@ suspend fun main() = coroutineScope { } ``` -> Play with coroutines online [here](https://pl.kotl.in/hG_tKbid_) +> Play with coroutines online [here](https://pl.kotl.in/9zva88r7S) ## Modules @@ -37,6 +37,7 @@ suspend fun main() = coroutineScope { * [core/jvm](kotlinx-coroutines-core/jvm/) — additional core features available on Kotlin/JVM: * [Dispatchers.IO] dispatcher for blocking coroutines; * [Executor.asCoroutineDispatcher][asCoroutineDispatcher] extension, custom thread pools, and more. + * Integrations with `CompletableFuture` and JVM-specific extensions. * [core/js](kotlinx-coroutines-core/js/) — additional core features available on Kotlin/JS: * Integration with `Promise` via [Promise.await] and [promise] builder; * Integration with `Window` via [Window.asCoroutineDispatcher], etc. @@ -56,7 +57,7 @@ suspend fun main() = coroutineScope { * [ui](ui/README.md) — modules that provide coroutine dispatchers for various single-threaded UI libraries: * Android, JavaFX, and Swing. * [integration](integration/README.md) — modules that provide integration with various asynchronous callback- and future-based libraries: - * JDK8 [CompletionStage.await], Guava [ListenableFuture.await], and Google Play Services [Task.await]; + * Guava [ListenableFuture.await], and Google Play Services [Task.await]; * SLF4J MDC integration via [MDCContext]. ## Documentation @@ -84,7 +85,7 @@ Add dependencies (you can also add other modules that you need): <dependency> <groupId>org.jetbrains.kotlinx</groupId> <artifactId>kotlinx-coroutines-core</artifactId> - <version>1.6.4</version> + <version>1.7.2</version> </dependency> ``` @@ -92,7 +93,7 @@ And make sure that you use the latest Kotlin version: ```xml <properties> - <kotlin.version>1.6.21</kotlin.version> + <kotlin.version>1.8.20</kotlin.version> </properties> ``` @@ -102,7 +103,7 @@ Add dependencies (you can also add other modules that you need): ```kotlin dependencies { - implementation("org.jetbrains.kotlinx:kotlinx-coroutines-core:1.6.4") + implementation("org.jetbrains.kotlinx:kotlinx-coroutines-core:1.7.2") } ``` @@ -111,10 +112,10 @@ And make sure that you use the latest Kotlin version: ```kotlin plugins { // For build.gradle.kts (Kotlin DSL) - kotlin("jvm") version "1.6.21" + kotlin("jvm") version "1.8.20" // For build.gradle (Groovy DSL) - id "org.jetbrains.kotlin.jvm" version "1.6.21" + id "org.jetbrains.kotlin.jvm" version "1.8.20" } ``` @@ -132,7 +133,7 @@ Add [`kotlinx-coroutines-android`](ui/kotlinx-coroutines-android) module as a dependency when using `kotlinx.coroutines` on Android: ```kotlin -implementation("org.jetbrains.kotlinx:kotlinx-coroutines-android:1.6.4") +implementation("org.jetbrains.kotlinx:kotlinx-coroutines-android:1.7.2") ``` This gives you access to the Android [Dispatchers.Main] @@ -167,7 +168,7 @@ In common code that should get compiled for different platforms, you can add a d ```kotlin commonMain { dependencies { - implementation("org.jetbrains.kotlinx:kotlinx-coroutines-core:1.6.4") + implementation("org.jetbrains.kotlinx:kotlinx-coroutines-core:1.7.2") } } ``` @@ -179,15 +180,15 @@ Platform-specific dependencies are recommended to be used only for non-multiplat #### JS Kotlin/JS version of `kotlinx.coroutines` is published as -[`kotlinx-coroutines-core-js`](https://search.maven.org/artifact/org.jetbrains.kotlinx/kotlinx-coroutines-core-js/1.6.4/jar) +[`kotlinx-coroutines-core-js`](https://central.sonatype.com/artifact/org.jetbrains.kotlinx/kotlinx-coroutines-core-js/1.7.2) (follow the link to get the dependency declaration snippet) and as [`kotlinx-coroutines-core`](https://www.npmjs.com/package/kotlinx-coroutines-core) NPM package. #### Native Kotlin/Native version of `kotlinx.coroutines` is published as -[`kotlinx-coroutines-core-$platform`](https://mvnrepository.com/search?q=kotlinx-coroutines-core-) where `$platform` is -the target Kotlin/Native platform. [List of currently supported targets](https://github.com/Kotlin/kotlinx.coroutines/blob/master/gradle/compile-native-multiplatform.gradle#L16). - +[`kotlinx-coroutines-core-$platform`](https://central.sonatype.com/search?q=kotlinx-coroutines-core&namespace=org.jetbrains.kotlinx) where `$platform` is +the target Kotlin/Native platform. +Targets are provided in accordance with [official K/N target support](https://kotlinlang.org/docs/native-target-support.html). ## Building and Contributing See [Contributing Guidelines](CONTRIBUTING.md). @@ -211,7 +212,7 @@ See [Contributing Guidelines](CONTRIBUTING.md). [MainScope()]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/-main-scope.html [SupervisorJob()]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/-supervisor-job.html [CoroutineExceptionHandler]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/-coroutine-exception-handler/index.html -[Dispatchers.IO]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/-dispatchers/-i-o.html +[Dispatchers.IO]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/-i-o.html [asCoroutineDispatcher]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/as-coroutine-dispatcher.html [Promise.await]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/await.html [promise]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/promise.html @@ -259,9 +260,6 @@ See [Contributing Guidelines](CONTRIBUTING.md). <!--- MODULE kotlinx-coroutines-jdk8 --> <!--- INDEX kotlinx.coroutines.future --> - -[CompletionStage.await]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-jdk8/kotlinx.coroutines.future/await.html - <!--- MODULE kotlinx-coroutines-guava --> <!--- INDEX kotlinx.coroutines.guava --> @@ -19,9 +19,10 @@ To release a new `<version>` of `kotlinx-coroutines`: * [`coroutines-guide-ui.md`](ui/coroutines-guide-ui.md) * Properties * [`gradle.properties`](gradle.properties) + * [`integration-testing/gradle.properties`](integration-testing/gradle.properties) * Make sure to **exclude** `CHANGES.md` from replacements. - As an alternative approach, you can use `./bump-version.sh old_version new_version` + As an alternative approach, you can use `./bump-version.sh new_version` 5. Write release notes in [`CHANGES.md`](CHANGES.md): * Use the old releases for style guidance. @@ -43,7 +44,7 @@ To release a new `<version>` of `kotlinx-coroutines`: * Get approval for it. 0. On [TeamCity integration server](https://teamcity.jetbrains.com/project.html?projectId=KotlinTools_KotlinxCoroutines): - * Wait until "Build" configuration for committed `master` branch passes tests. + * Wait until "Build" configuration for committed `version-<version>` branch passes tests. * Run "Deploy (Configure, RUN THIS ONE)" configuration with the corresponding new version: - Use the `version-<version>` branch - Set the `DeployVersion` build parameter to `<version>` @@ -76,6 +77,5 @@ To release a new `<version>` of `kotlinx-coroutines`: 8. Push the updates to GitHub:<br> `git push` -9. Build and publish the documentation for the web-site: <br> - * Set new value for [`kotlinx.coroutines.release.tag`](https://buildserver.labs.intellij.net/admin/editProject.html?projectId=Kotlin_KotlinSites_Builds_KotlinlangOrg_LibrariesAPIs&tab=projectParams) - * And run deploy [configuration](https://buildserver.labs.intellij.net/buildConfiguration/Kotlin_KotlinSites_Builds_KotlinlangOrg_KotlinCoroutinesApi?branch=%3Cdefault%3E&buildTypeTab=overview&mode=builds) +9. Propose the website documentation update: <br> + * Set new value for [`KOTLINX_COROUTINES_RELEASE_TAG`](https://github.com/JetBrains/kotlin-web-site/blob/master/.teamcity/BuildParams.kt), creating a Pull Request in the website's repository. diff --git a/benchmarks/build.gradle.kts b/benchmarks/build.gradle.kts index f64c4aaa..e64f1890 100644 --- a/benchmarks/build.gradle.kts +++ b/benchmarks/build.gradle.kts @@ -4,12 +4,12 @@ @file:Suppress("UnstableApiUsage") -import me.champeau.gradle.* +import me.champeau.jmh.* import org.jetbrains.kotlin.gradle.tasks.* plugins { id("com.github.johnrengelman.shadow") - id("me.champeau.gradle.jmh") apply false + id("me.champeau.jmh") } repositories { @@ -21,33 +21,19 @@ java { targetCompatibility = JavaVersion.VERSION_1_8 } -apply(plugin="me.champeau.gradle.jmh") - tasks.named<KotlinCompile>("compileJmhKotlin") { kotlinOptions { jvmTarget = "1.8" - freeCompilerArgs += "-Xjvm-default=enable" - } -} - -// It is better to use the following to run benchmarks, otherwise you may get unexpected errors: -// ./gradlew --no-daemon cleanJmhJar jmh -Pjmh="MyBenchmark" -extensions.configure<JMHPluginExtension>("jmh") { - jmhVersion = "1.26" - duplicateClassesStrategy = DuplicatesStrategy.INCLUDE - failOnError = true - resultFormat = "CSV" - project.findProperty("jmh")?.also { - include = listOf(".*$it.*") + freeCompilerArgs += "-Xjvm-default=all" } -// includeTests = false } val jmhJarTask = tasks.named<Jar>("jmhJar") { archiveBaseName by "benchmarks" archiveClassifier by null archiveVersion by null - destinationDirectory.file("$rootDir") + archiveVersion.convention(null as String?) + destinationDirectory.set(file("$rootDir")) } tasks { @@ -63,13 +49,14 @@ tasks { } dependencies { - implementation("org.openjdk.jmh:jmh-core:1.26") + implementation("org.openjdk.jmh:jmh-core:1.35") implementation("io.projectreactor:reactor-core:${version("reactor")}") implementation("io.reactivex.rxjava2:rxjava:2.1.9") implementation("com.github.akarnokd:rxjava2-extensions:0.20.8") implementation("com.typesafe.akka:akka-actor_2.12:2.5.0") implementation(project(":kotlinx-coroutines-core")) + implementation(project(":kotlinx-coroutines-debug")) implementation(project(":kotlinx-coroutines-reactive")) // add jmh dependency on main diff --git a/benchmarks/src/jmh/kotlin/benchmarks/ChannelProducerConsumerBenchmark.kt b/benchmarks/src/jmh/kotlin/benchmarks/ChannelProducerConsumerBenchmark.kt index 0fa50489..0aa218e8 100644 --- a/benchmarks/src/jmh/kotlin/benchmarks/ChannelProducerConsumerBenchmark.kt +++ b/benchmarks/src/jmh/kotlin/benchmarks/ChannelProducerConsumerBenchmark.kt @@ -4,15 +4,14 @@ package benchmarks +import benchmarks.common.* import kotlinx.coroutines.* import kotlinx.coroutines.channels.Channel +import kotlinx.coroutines.scheduling.* import kotlinx.coroutines.selects.select import org.openjdk.jmh.annotations.* -import org.openjdk.jmh.infra.Blackhole import java.lang.Integer.max -import java.util.concurrent.ForkJoinPool import java.util.concurrent.Phaser -import java.util.concurrent.ThreadLocalRandom import java.util.concurrent.TimeUnit @@ -26,14 +25,14 @@ import java.util.concurrent.TimeUnit * Please, be patient, this benchmark takes quite a lot of time to complete. */ @Warmup(iterations = 3, time = 500, timeUnit = TimeUnit.MICROSECONDS) -@Measurement(iterations = 10, time = 500, timeUnit = TimeUnit.MICROSECONDS) -@Fork(value = 3) -@BenchmarkMode(Mode.AverageTime) +@Measurement(iterations = 20, time = 500, timeUnit = TimeUnit.MICROSECONDS) +@Fork(value = 1) +@BenchmarkMode(Mode.Throughput) @OutputTimeUnit(TimeUnit.MILLISECONDS) @State(Scope.Benchmark) open class ChannelProducerConsumerBenchmark { @Param - private var _0_dispatcher: DispatcherCreator = DispatcherCreator.FORK_JOIN + private var _0_dispatcher: DispatcherCreator = DispatcherCreator.DEFAULT @Param private var _1_channel: ChannelCreator = ChannelCreator.RENDEZVOUS @@ -44,12 +43,13 @@ open class ChannelProducerConsumerBenchmark { @Param("false", "true") private var _3_withSelect: Boolean = false - @Param("1", "2", "4") // local machine -// @Param("1", "2", "4", "8", "12") // local machine -// @Param("1", "2", "4", "8", "16", "32", "64", "128", "144") // dasquad -// @Param("1", "2", "4", "8", "16", "32", "64", "96") // Google Cloud + @Param("1", "2", "4", "8", "16") // local machine +// @Param("1", "2", "4", "8", "16", "32", "64", "128") // Server private var _4_parallelism: Int = 0 + @Param("50") + private var _5_workSize: Int = 0 + private lateinit var dispatcher: CoroutineDispatcher private lateinit var channel: Channel<Int> @@ -61,7 +61,7 @@ open class ChannelProducerConsumerBenchmark { } @Benchmark - fun spmc() { + fun mcsp() { if (_2_coroutines != 0) return val producers = max(1, _4_parallelism - 1) val consumers = 1 @@ -69,6 +69,14 @@ open class ChannelProducerConsumerBenchmark { } @Benchmark + fun spmc() { + if (_2_coroutines != 0) return + val producers = 1 + val consumers = max(1, _4_parallelism - 1) + run(producers, consumers) + } + + @Benchmark fun mpmc() { val producers = if (_2_coroutines == 0) (_4_parallelism + 1) / 2 else _2_coroutines / 2 val consumers = producers @@ -76,7 +84,7 @@ open class ChannelProducerConsumerBenchmark { } private fun run(producers: Int, consumers: Int) { - val n = APPROX_BATCH_SIZE / producers * producers + val n = (APPROX_BATCH_SIZE / producers * producers) / consumers * consumers val phaser = Phaser(producers + consumers + 1) // Run producers repeat(producers) { @@ -111,7 +119,7 @@ open class ChannelProducerConsumerBenchmark { } else { channel.send(element) } - doWork() + doWork(_5_workSize) } private suspend fun consume(dummy: Channel<Int>?) { @@ -123,28 +131,25 @@ open class ChannelProducerConsumerBenchmark { } else { channel.receive() } - doWork() + doWork(_5_workSize) } } enum class DispatcherCreator(val create: (parallelism: Int) -> CoroutineDispatcher) { - FORK_JOIN({ parallelism -> ForkJoinPool(parallelism).asCoroutineDispatcher() }) + //FORK_JOIN({ parallelism -> ForkJoinPool(parallelism).asCoroutineDispatcher() }), + @Suppress("INVISIBLE_MEMBER", "INVISIBLE_REFERENCE") + DEFAULT({ parallelism -> ExperimentalCoroutineDispatcher(corePoolSize = parallelism, maxPoolSize = parallelism) }) } enum class ChannelCreator(private val capacity: Int) { RENDEZVOUS(Channel.RENDEZVOUS), -// BUFFERED_1(1), - BUFFERED_2(2), -// BUFFERED_4(4), - BUFFERED_32(32), - BUFFERED_128(128), + BUFFERED_16(16), + BUFFERED_64(64), BUFFERED_UNLIMITED(Channel.UNLIMITED); fun create(): Channel<Int> = Channel(capacity) } -private fun doWork(): Unit = Blackhole.consumeCPU(ThreadLocalRandom.current().nextLong(WORK_MIN, WORK_MAX)) +private fun doWork(workSize: Int): Unit = doGeomDistrWork(workSize) -private const val WORK_MIN = 50L -private const val WORK_MAX = 100L -private const val APPROX_BATCH_SIZE = 100000 +private const val APPROX_BATCH_SIZE = 100_000 diff --git a/benchmarks/src/jmh/kotlin/benchmarks/ChannelSinkNoAllocationsBenchmark.kt b/benchmarks/src/jmh/kotlin/benchmarks/ChannelSinkNoAllocationsBenchmark.kt new file mode 100644 index 00000000..dcba8383 --- /dev/null +++ b/benchmarks/src/jmh/kotlin/benchmarks/ChannelSinkNoAllocationsBenchmark.kt @@ -0,0 +1,37 @@ +/* + * Copyright 2016-2023 JetBrains s.r.o. Use of this source code is governed by the Apache 2.0 license. + */ + +package benchmarks + +import kotlinx.coroutines.* +import kotlinx.coroutines.channels.* +import org.openjdk.jmh.annotations.* +import java.util.concurrent.* +import kotlin.coroutines.* + +@Warmup(iterations = 3, time = 1) +@Measurement(iterations = 5, time = 1) +@BenchmarkMode(Mode.AverageTime) +@OutputTimeUnit(TimeUnit.MILLISECONDS) +@State(Scope.Benchmark) +@Fork(1) +open class ChannelSinkNoAllocationsBenchmark { + private val unconfined = Dispatchers.Unconfined + + @Benchmark + fun channelPipeline(): Int = runBlocking { + run(unconfined) + } + + private suspend inline fun run(context: CoroutineContext): Int { + var size = 0 + Channel.range(context).consumeEach { size++ } + return size + } + + private fun Channel.Factory.range(context: CoroutineContext) = GlobalScope.produce(context) { + for (i in 0 until 100_000) + send(Unit) // no allocations + } +} diff --git a/benchmarks/src/jmh/kotlin/benchmarks/SemaphoreBenchmark.kt b/benchmarks/src/jmh/kotlin/benchmarks/SemaphoreBenchmark.kt index 9e1bfc43..6826b7a1 100644 --- a/benchmarks/src/jmh/kotlin/benchmarks/SemaphoreBenchmark.kt +++ b/benchmarks/src/jmh/kotlin/benchmarks/SemaphoreBenchmark.kt @@ -7,6 +7,7 @@ package benchmarks import benchmarks.common.* import kotlinx.coroutines.* import kotlinx.coroutines.channels.* +import kotlinx.coroutines.scheduling.* import kotlinx.coroutines.sync.* import org.openjdk.jmh.annotations.* import java.util.concurrent.* @@ -19,7 +20,7 @@ import java.util.concurrent.* @State(Scope.Benchmark) open class SemaphoreBenchmark { @Param - private var _1_dispatcher: SemaphoreBenchDispatcherCreator = SemaphoreBenchDispatcherCreator.FORK_JOIN + private var _1_dispatcher: SemaphoreBenchDispatcherCreator = SemaphoreBenchDispatcherCreator.DEFAULT @Param("0", "1000") private var _2_coroutines: Int = 0 @@ -27,9 +28,8 @@ open class SemaphoreBenchmark { @Param("1", "2", "4", "8", "32", "128", "100000") private var _3_maxPermits: Int = 0 - @Param("1", "2", "4") // local machine -// @Param("1", "2", "4", "8", "16", "32", "64", "128", "144") // dasquad -// @Param("1", "2", "4", "8", "16", "32", "64", "96") // Google Cloud + @Param("1", "2", "4", "8", "16") // local machine +// @Param("1", "2", "4", "8", "16", "32", "64", "128") // Server private var _4_parallelism: Int = 0 private lateinit var dispatcher: CoroutineDispatcher @@ -80,10 +80,11 @@ open class SemaphoreBenchmark { } enum class SemaphoreBenchDispatcherCreator(val create: (parallelism: Int) -> CoroutineDispatcher) { - FORK_JOIN({ parallelism -> ForkJoinPool(parallelism).asCoroutineDispatcher() }), - EXPERIMENTAL({ parallelism -> Dispatchers.Default }) // TODO doesn't take parallelism into account + // FORK_JOIN({ parallelism -> ForkJoinPool(parallelism).asCoroutineDispatcher() }), + @Suppress("INVISIBLE_MEMBER", "INVISIBLE_REFERENCE") + DEFAULT({ parallelism -> ExperimentalCoroutineDispatcher(corePoolSize = parallelism, maxPoolSize = parallelism) }) } -private const val WORK_INSIDE = 80 -private const val WORK_OUTSIDE = 40 -private const val BATCH_SIZE = 1000000 +private const val WORK_INSIDE = 50 +private const val WORK_OUTSIDE = 50 +private const val BATCH_SIZE = 100000 diff --git a/benchmarks/src/jmh/kotlin/benchmarks/SequentialSemaphoreBenchmark.kt b/benchmarks/src/jmh/kotlin/benchmarks/SequentialSemaphoreBenchmark.kt new file mode 100644 index 00000000..6926db78 --- /dev/null +++ b/benchmarks/src/jmh/kotlin/benchmarks/SequentialSemaphoreBenchmark.kt @@ -0,0 +1,43 @@ +/* + * Copyright 2016-2021 JetBrains s.r.o. Use of this source code is governed by the Apache 2.0 license. + */ + +package benchmarks + +import kotlinx.coroutines.* +import kotlinx.coroutines.sync.* +import org.openjdk.jmh.annotations.* +import java.util.concurrent.TimeUnit +import kotlin.test.* + +@Warmup(iterations = 5, time = 1) +@Measurement(iterations = 10, time = 1) +@BenchmarkMode(Mode.AverageTime) +@OutputTimeUnit(TimeUnit.MILLISECONDS) +@State(Scope.Benchmark) +@Fork(1) +open class SequentialSemaphoreAsMutexBenchmark { + val s = Semaphore(1) + + @Benchmark + fun benchmark() : Unit = runBlocking { + val s = Semaphore(permits = 1, acquiredPermits = 1) + var step = 0 + launch(Dispatchers.Unconfined) { + repeat(N) { + assertEquals(it * 2, step) + step++ + s.acquire() + } + } + repeat(N) { + assertEquals(it * 2 + 1, step) + step++ + s.release() + } + } +} + +fun main() = SequentialSemaphoreAsMutexBenchmark().benchmark() + +private val N = 1_000_000
\ No newline at end of file diff --git a/benchmarks/src/jmh/kotlin/benchmarks/debug/DebugSequenceOverheadBenchmark.kt b/benchmarks/src/jmh/kotlin/benchmarks/debug/DebugSequenceOverheadBenchmark.kt new file mode 100644 index 00000000..16e93e1f --- /dev/null +++ b/benchmarks/src/jmh/kotlin/benchmarks/debug/DebugSequenceOverheadBenchmark.kt @@ -0,0 +1,90 @@ +/* + * Copyright 2016-2023 JetBrains s.r.o. Use of this source code is governed by the Apache 2.0 license. + */ + +package benchmarks.debug + +import kotlinx.coroutines.* +import kotlinx.coroutines.debug.* +import org.openjdk.jmh.annotations.* +import org.openjdk.jmh.annotations.State +import java.util.concurrent.* +import java.util.concurrent.atomic.AtomicInteger + +/** + * The benchmark is supposed to show the DebugProbes overhead for a non-concurrent sequence builder. + * The code is actually part of the IDEA codebase, originally reported here: https://github.com/Kotlin/kotlinx.coroutines/issues/3527 + */ +@Warmup(iterations = 5, time = 1) +@Measurement(iterations = 5, time = 1) +@Fork(value = 1) +@BenchmarkMode(Mode.AverageTime) +@OutputTimeUnit(TimeUnit.MICROSECONDS) +@State(Scope.Benchmark) +open class DebugSequenceOverheadBenchmark { + + private fun <Node> generateRecursiveSequence( + initialSequence: Sequence<Node>, + children: (Node) -> Sequence<Node> + ): Sequence<Node> { + return sequence { + val initialIterator = initialSequence.iterator() + if (!initialIterator.hasNext()) { + return@sequence + } + val visited = HashSet<Node>() + val sequences = ArrayDeque<Sequence<Node>>() + sequences.addLast(initialIterator.asSequence()) + while (sequences.isNotEmpty()) { + val currentSequence = sequences.removeFirst() + for (node in currentSequence) { + if (visited.add(node)) { + yield(node) + sequences.addLast(children(node)) + } + } + } + } + } + + @Param("true", "false") + var withDebugger = false + + @Setup + fun setup() { + DebugProbes.sanitizeStackTraces = false + DebugProbes.enableCreationStackTraces = false + if (withDebugger) { + DebugProbes.install() + } + } + + @TearDown + fun tearDown() { + if (withDebugger) { + DebugProbes.uninstall() + } + } + + // Shows the overhead of sequence builder with debugger enabled + @Benchmark + fun runSequenceSingleThread(): Int = runBlocking { + generateRecursiveSequence((1..100).asSequence()) { + (1..it).asSequence() + }.sum() + } + + // Shows the overhead of sequence builder with debugger enabled and debugger is concurrently stressed out + @Benchmark + fun runSequenceMultipleThreads(): Int = runBlocking { + val result = AtomicInteger(0) + repeat(Runtime.getRuntime().availableProcessors()) { + launch(Dispatchers.Default) { + result.addAndGet(generateRecursiveSequence((1..100).asSequence()) { + (1..it).asSequence() + }.sum()) + } + } + result.get() + } +} diff --git a/benchmarks/src/jmh/kotlin/benchmarks/flow/scrabble/ShakespearePlaysScrabble.kt b/benchmarks/src/jmh/kotlin/benchmarks/flow/scrabble/ShakespearePlaysScrabble.kt index 006d36c0..10433fcb 100644 --- a/benchmarks/src/jmh/kotlin/benchmarks/flow/scrabble/ShakespearePlaysScrabble.kt +++ b/benchmarks/src/jmh/kotlin/benchmarks/flow/scrabble/ShakespearePlaysScrabble.kt @@ -34,14 +34,12 @@ abstract class ShakespearePlaysScrabble { public interface LongWrapper { fun get(): Long - @JvmDefault fun incAndSet(): LongWrapper { return object : LongWrapper { override fun get(): Long = this@LongWrapper.get() + 1L } } - @JvmDefault fun add(other: LongWrapper): LongWrapper { return object : LongWrapper { override fun get(): Long = this@LongWrapper.get() + other.get() diff --git a/benchmarks/src/jmh/kotlin/benchmarks/tailcall/SelectBenchmark.kt b/benchmarks/src/jmh/kotlin/benchmarks/tailcall/SelectBenchmark.kt new file mode 100644 index 00000000..cb4d39ee --- /dev/null +++ b/benchmarks/src/jmh/kotlin/benchmarks/tailcall/SelectBenchmark.kt @@ -0,0 +1,42 @@ +/* + * Copyright 2016-2022 JetBrains s.r.o. Use of this source code is governed by the Apache 2.0 license. + */ + +package benchmarks.tailcall + +import kotlinx.coroutines.* +import kotlinx.coroutines.channels.* +import kotlinx.coroutines.selects.* +import org.openjdk.jmh.annotations.* +import java.util.concurrent.* + +@Warmup(iterations = 8, time = 1, timeUnit = TimeUnit.SECONDS) +@Measurement(iterations = 8, time = 1, timeUnit = TimeUnit.SECONDS) +@Fork(value = 1) +@BenchmarkMode(Mode.AverageTime) +@OutputTimeUnit(TimeUnit.MICROSECONDS) +@State(Scope.Benchmark) +open class SelectBenchmark { + // 450 + private val iterations = 1000 + + @Benchmark + fun stressSelect() = runBlocking { + val pingPong = Channel<Unit>() + launch { + repeat(iterations) { + select { + pingPong.onSend(Unit) {} + } + } + } + + launch { + repeat(iterations) { + select { + pingPong.onReceive() {} + } + } + } + } +} diff --git a/build.gradle b/build.gradle index 4d6af165..84d77701 100644 --- a/build.gradle +++ b/build.gradle @@ -5,7 +5,10 @@ import org.jetbrains.kotlin.config.KotlinCompilerVersion import org.jetbrains.kotlin.gradle.plugin.KotlinPlatformType -import org.jetbrains.kotlin.gradle.tasks.AbstractKotlinCompile +import org.jetbrains.kotlin.gradle.targets.js.nodejs.NodeJsRootExtension +import org.jetbrains.kotlin.gradle.targets.js.nodejs.NodeJsRootPlugin +import org.jetbrains.kotlin.gradle.targets.js.yarn.YarnPlugin +import org.jetbrains.kotlin.gradle.targets.js.yarn.YarnRootExtension import org.jetbrains.kotlin.konan.target.HostManager import org.jetbrains.dokka.gradle.DokkaTaskPartial @@ -42,10 +45,16 @@ buildscript { } } + if (using_snapshot_version) { + repositories { + mavenLocal() + } + } + repositories { mavenCentral() maven { url "https://plugins.gradle.org/m2/" } - maven { url "https://maven.pkg.jetbrains.space/kotlin/p/kotlin/dev" } + CommunityProjectsBuild.addDevRepositoryIfEnabled(delegate, project) mavenLocal() } @@ -57,7 +66,8 @@ buildscript { classpath "com.github.node-gradle:gradle-node-plugin:$gradle_node_version" classpath "org.jetbrains.kotlinx:binary-compatibility-validator:$binary_compatibility_validator_version" classpath "ru.vyarus:gradle-animalsniffer-plugin:1.5.4" // Android API check - classpath "org.jetbrains.kotlinx:kover:$kover_version" + classpath "org.jetbrains.kotlin:atomicfu:$kotlin_version" + classpath "org.jetbrains.kotlinx:kover-gradle-plugin:$kover_version" // JMH plugins classpath "gradle.plugin.com.github.johnrengelman:shadow:7.1.2" @@ -75,6 +85,9 @@ def configureKotlinJvmPlatform(configuration) { configuration.attributes.attribute(KotlinPlatformType.attribute, KotlinPlatformType.jvm) } +// Configure subprojects with Kotlin sources +apply plugin: "configure-compilation-conventions" + allprojects { // the only place where HostManager could be instantiated project.ext.hostManager = new HostManager() @@ -93,6 +106,12 @@ allprojects { kotlin_version = rootProject.properties['kotlin_snapshot_version'] } + if (using_snapshot_version) { + repositories { + mavenLocal() + } + } + ext.unpublished = unpublished // This project property is set during nightly stress test @@ -126,7 +145,7 @@ allprojects { */ google() mavenCentral() - maven { url "https://maven.pkg.jetbrains.space/kotlin/p/kotlin/dev" } + CommunityProjectsBuild.addDevRepositoryIfEnabled(delegate, project) } } @@ -165,22 +184,7 @@ configure(subprojects.findAll { !sourceless.contains(it.name) && it.name != core } apply plugin: "bom-conventions" - -// Configure subprojects with Kotlin sources -configure(subprojects.findAll { !sourceless.contains(it.name) }) { - // Use atomicfu plugin, it also adds all the necessary dependencies - apply plugin: 'kotlinx-atomicfu' - - // Configure options for all Kotlin compilation tasks - tasks.withType(AbstractKotlinCompile).all { - kotlinOptions.freeCompilerArgs += OptInPreset.optInAnnotations.collect { "-Xopt-in=" + it } - kotlinOptions.freeCompilerArgs += "-progressive" - // Disable KT-36770 for RxJava2 integration - kotlinOptions.freeCompilerArgs += "-XXLanguage:-ProhibitUsingNullableTypeParameterAgainstNotNullAnnotated" - // Remove null assertions to get smaller bytecode on Android - kotlinOptions.freeCompilerArgs += ["-Xno-param-assertions", "-Xno-receiver-assertions", "-Xno-call-assertions"] - } -} +apply plugin: "java-modularity-conventions" if (build_snapshot_train) { println "Hacking test tasks, removing stress and flaky tests" @@ -235,8 +239,9 @@ def core_docs_url = "https://kotlinlang.org/api/kotlinx.coroutines/$coreModule/" def core_docs_file = "$projectDir/kotlinx-coroutines-core/build/dokka/htmlPartial/package-list" apply plugin: "org.jetbrains.dokka" -configure(subprojects.findAll { !unpublished.contains(it.name) && it.name != coreModule }) { - if (it.name != 'kotlinx-coroutines-bom') { +configure(subprojects.findAll { !unpublished.contains(it.name) + && it.name != coreModule }) { + if (it.name != 'kotlinx-coroutines-bom' && it.name != jdk8ObsoleteModule) { apply from: rootProject.file('gradle/dokka.gradle.kts') } apply from: rootProject.file('gradle/publish.gradle') @@ -244,7 +249,7 @@ configure(subprojects.findAll { !unpublished.contains(it.name) && it.name != cor configure(subprojects.findAll { !unpublished.contains(it.name) }) { if (it.name != "kotlinx-coroutines-bom") { - if (it.name != coreModule) { + if (it.name != coreModule && it.name != jdk8ObsoleteModule) { tasks.withType(DokkaTaskPartial.class) { dokkaSourceSets.configureEach { externalDocumentationLink { @@ -355,3 +360,19 @@ allprojects { subProject -> } } } + +tasks.named("dokkaHtmlMultiModule") { + pluginsMapConfiguration.set(["org.jetbrains.dokka.base.DokkaBase": """{ "templatesDir": "${projectDir.toString().replace('\\', '/')}/dokka-templates" }"""]) +} + +if (CacheRedirector.enabled) { + def yarnRootExtension = rootProject.extensions.findByType(YarnRootExtension.class) + if (yarnRootExtension != null) { + yarnRootExtension.downloadBaseUrl = CacheRedirector.maybeRedirect(yarnRootExtension.downloadBaseUrl) + } + + def nodeJsExtension = rootProject.extensions.findByType(NodeJsRootExtension.class) + if (nodeJsExtension != null) { + nodeJsExtension.nodeDownloadBaseUrl = CacheRedirector.maybeRedirect(nodeJsExtension.nodeDownloadBaseUrl) + } +} diff --git a/buildSrc/build.gradle.kts b/buildSrc/build.gradle.kts index eaa03f2f..ae54ad0f 100644 --- a/buildSrc/build.gradle.kts +++ b/buildSrc/build.gradle.kts @@ -10,6 +10,7 @@ plugins { val cacheRedirectorEnabled = System.getenv("CACHE_REDIRECTOR")?.toBoolean() == true val buildSnapshotTrain = properties["build_snapshot_train"]?.toString()?.toBoolean() == true +val kotlinDevUrl = project.rootProject.properties["kotlin_repo_url"] as? String repositories { mavenCentral() @@ -18,17 +19,15 @@ repositories { } else { maven("https://plugins.gradle.org/m2") } - maven("https://maven.pkg.jetbrains.space/kotlin/p/kotlin/dev") + if (!kotlinDevUrl.isNullOrEmpty()) { + maven(kotlinDevUrl) + } if (buildSnapshotTrain) { mavenLocal() } } -kotlinDslPluginOptions { - experimentalWarning.set(false) -} - -val props = Properties().apply { +val gradleProperties = Properties().apply { file("../gradle.properties").inputStream().use { load(it) } } @@ -38,7 +37,9 @@ fun version(target: String): String { val snapshotVersion = properties["kotlin_snapshot_version"] if (snapshotVersion != null) return snapshotVersion.toString() } - return props.getProperty("${target}_version") + val version = "${target}_version" + // Read from CLI first, used in aggregate builds + return properties[version]?.let{"$it"} ?: gradleProperties.getProperty(version) } dependencies { @@ -59,7 +60,7 @@ dependencies { exclude(group = "org.jetbrains.kotlin", module = "kotlin-stdlib") } implementation("ru.vyarus:gradle-animalsniffer-plugin:1.5.3") // Android API check - implementation("org.jetbrains.kotlinx:kover:${version("kover")}") { + implementation("org.jetbrains.kotlinx:kover-gradle-plugin:${version("kover")}") { exclude(group = "org.jetbrains.kotlin", module = "kotlin-stdlib-jdk8") exclude(group = "org.jetbrains.kotlin", module = "kotlin-stdlib-jdk7") exclude(group = "org.jetbrains.kotlin", module = "kotlin-stdlib") diff --git a/buildSrc/src/main/kotlin/CacheRedirector.kt b/buildSrc/src/main/kotlin/CacheRedirector.kt index bcadd736..c0d83515 100644 --- a/buildSrc/src/main/kotlin/CacheRedirector.kt +++ b/buildSrc/src/main/kotlin/CacheRedirector.kt @@ -26,31 +26,33 @@ private val mirroredUrls = listOf( "https://dl.google.com/dl/android/studio/ide-zips", "https://dl.google.com/go", "https://download.jetbrains.com", + "https://github.com/yarnpkg/yarn/releases/download", "https://jitpack.io", - "https://maven.pkg.jetbrains.space/kotlin/p/kotlin/dev", "https://maven.pkg.jetbrains.space/kotlin/p/kotlin/bootstrap", + "https://maven.pkg.jetbrains.space/kotlin/p/kotlin/dev", "https://maven.pkg.jetbrains.space/kotlin/p/kotlin/eap", + "https://nodejs.org/dist", "https://oss.sonatype.org/content/repositories/releases", "https://oss.sonatype.org/content/repositories/snapshots", "https://oss.sonatype.org/content/repositories/staging", "https://packages.confluent.io/maven/", "https://plugins.gradle.org/m2", "https://plugins.jetbrains.com/maven", - "https://repo1.maven.org/maven2", "https://repo.grails.org/grails/core", "https://repo.jenkins-ci.org/releases", "https://repo.maven.apache.org/maven2", "https://repo.spring.io/milestone", "https://repo.typesafe.com/typesafe/ivy-releases", + "https://repo1.maven.org/maven2", "https://services.gradle.org", "https://www.exasol.com/artifactory/exasol-releases", + "https://www.jetbrains.com/intellij-repository/nightly", + "https://www.jetbrains.com/intellij-repository/releases", + "https://www.jetbrains.com/intellij-repository/snapshots", "https://www.myget.org/F/intellij-go-snapshots/maven", "https://www.myget.org/F/rd-model-snapshots/maven", "https://www.myget.org/F/rd-snapshots/maven", "https://www.python.org/ftp", - "https://www.jetbrains.com/intellij-repository/nightly", - "https://www.jetbrains.com/intellij-repository/releases", - "https://www.jetbrains.com/intellij-repository/snapshots" ) private val aliases = mapOf( @@ -115,4 +117,13 @@ object CacheRedirector { fun Project.configure() { checkRedirect(repositories, displayName) } + + @JvmStatic + fun maybeRedirect(url: String): String { + if (!cacheRedirectorEnabled) return url + return URI(url).maybeRedirect()?.toString() ?: url + } + + @JvmStatic + val isEnabled get() = cacheRedirectorEnabled } diff --git a/buildSrc/src/main/kotlin/CommunityProjectsBuild.kt b/buildSrc/src/main/kotlin/CommunityProjectsBuild.kt new file mode 100644 index 00000000..155c9e48 --- /dev/null +++ b/buildSrc/src/main/kotlin/CommunityProjectsBuild.kt @@ -0,0 +1,72 @@ +/* + * Copyright 2016-2022 JetBrains s.r.o. Use of this source code is governed by the Apache 2.0 license. + */ +@file:JvmName("CommunityProjectsBuild") + +import org.gradle.api.* +import org.gradle.api.artifacts.dsl.* +import java.net.* +import java.util.logging.* + +private val LOGGER: Logger = Logger.getLogger("Kotlin settings logger") + + +/** + * Functions in this file are responsible for configuring kotlinx.coroutines build against a custom dev version + * of Kotlin compiler. + * Such configuration is used in a composite community build of Kotlin in order to check whether not-yet-released changes + * are compatible with our libraries (aka "integration testing that substitues lack of unit testing"). + */ + +/** + * Should be used for running against of non-released Kotlin compiler on a system test level. + * + * @return a Kotlin API version parametrized from command line nor gradle.properties, null otherwise + */ +fun getOverriddenKotlinApiVersion(project: Project): String? { + val apiVersion = project.rootProject.properties["kotlin_api_version"] as? String + if (apiVersion != null) { + LOGGER.info("""Configured Kotlin API version: '$apiVersion' for project $${project.name}""") + } + return apiVersion +} + +/** + * Should be used for running against of non-released Kotlin compiler on a system test level + * + * @return a Kotlin Language version parametrized from command line nor gradle.properties, null otherwise + */ +fun getOverriddenKotlinLanguageVersion(project: Project): String? { + val languageVersion = project.rootProject.properties["kotlin_language_version"] as? String + if (languageVersion != null) { + LOGGER.info("""Configured Kotlin Language version: '$languageVersion' for project ${project.name}""") + } + return languageVersion +} + +/** + * Should be used for running against of non-released Kotlin compiler on a system test level + * Kotlin compiler artifacts are expected to be downloaded from maven central by default. + * In case of compiling with not-published into the MC kotlin compiler artifacts, a kotlin_repo_url gradle parameter should be specified. + * To reproduce a build locally, a kotlin/dev repo should be passed + * + * @return an url for a kotlin compiler repository parametrized from command line nor gradle.properties, empty string otherwise + */ +fun getKotlinDevRepositoryUrl(project: Project): URI? { + val url: String? = project.rootProject.properties["kotlin_repo_url"] as? String + if (url != null) { + LOGGER.info("""Configured Kotlin Compiler repository url: '$url' for project ${project.name}""") + return URI.create(url) + } + return null +} + +/** + * Adds a kotlin-dev space repository with dev versions of Kotlin if Kotlin aggregate build is enabled + */ +fun addDevRepositoryIfEnabled(rh: RepositoryHandler, project: Project) { + val devRepoUrl = getKotlinDevRepositoryUrl(project) ?: return + rh.maven { + url = devRepoUrl + } +} diff --git a/buildSrc/src/main/kotlin/Java9Modularity.kt b/buildSrc/src/main/kotlin/Java9Modularity.kt new file mode 100644 index 00000000..ccd7ef33 --- /dev/null +++ b/buildSrc/src/main/kotlin/Java9Modularity.kt @@ -0,0 +1,152 @@ +/* + * Copyright 2016-2022 JetBrains s.r.o. Use of this source code is governed by the Apache 2.0 license. + */ + +import org.gradle.api.* +import org.gradle.api.attributes.* +import org.gradle.api.file.* +import org.gradle.api.tasks.* +import org.gradle.api.tasks.bundling.* +import org.gradle.api.tasks.compile.* +import org.gradle.jvm.toolchain.* +import org.gradle.kotlin.dsl.* +import org.gradle.work.* +import org.jetbrains.kotlin.gradle.dsl.* + +/** + * This object configures the Java compilation of a JPMS (aka Jigsaw) module descriptor. + * The source file for the module descriptor is expected at <project-dir>/src/module-info.java. + * + * To maintain backwards compatibility with Java 8, the jvm JAR is marked as a multi-release JAR + * with the module-info.class being moved to META-INF/versions/9/module-info.class. + * + * The Java toolchains feature of Gradle is used to detect or provision a JDK 11, + * which is used to compile the module descriptor. + */ +object Java9Modularity { + + /** + * Task that patches `module-info.java` and removes `requires kotlinx.atomicfu` directive. + * + * To have JPMS properly supported, Kotlin compiler **must** be supplied with the correct `module-info.java`. + * The correct module info has to contain `atomicfu` requirement because atomicfu plugin kicks-in **after** + * the compilation process. But `atomicfu` is compile-only dependency that shouldn't be present in the final + * `module-info.java` and that's exactly what this task ensures. + */ + abstract class ProcessModuleInfoFile : DefaultTask() { + @get:InputFile + @get:NormalizeLineEndings + abstract val moduleInfoFile: RegularFileProperty + + @get:OutputFile + abstract val processedModuleInfoFile: RegularFileProperty + + private val projectPath = project.path + + @TaskAction + fun process() { + val sourceFile = moduleInfoFile.get().asFile + if (!sourceFile.exists()) { + throw IllegalStateException("$sourceFile not found in $projectPath") + } + val outputFile = processedModuleInfoFile.get().asFile + sourceFile.useLines { lines -> + outputFile.outputStream().bufferedWriter().use { writer -> + for (line in lines) { + if ("kotlinx.atomicfu" in line) continue + writer.write(line) + writer.newLine() + } + } + } + } + } + + @JvmStatic + fun configure(project: Project) = with(project) { + val javaToolchains = extensions.findByType(JavaToolchainService::class.java) + ?: error("Gradle JavaToolchainService is not available") + val target = when (val kotlin = extensions.getByName("kotlin")) { + is KotlinJvmProjectExtension -> kotlin.target + is KotlinMultiplatformExtension -> kotlin.targets.getByName("jvm") + else -> throw IllegalStateException("Unknown Kotlin project extension in $project") + } + val compilation = target.compilations.getByName("main") + + // Force the use of JARs for compile dependencies, so any JPMS descriptors are picked up. + // For more details, see https://github.com/gradle/gradle/issues/890#issuecomment-623392772 + configurations.getByName(compilation.compileDependencyConfigurationName).attributes { + attribute( + LibraryElements.LIBRARY_ELEMENTS_ATTRIBUTE, + objects.named(LibraryElements::class, LibraryElements.JAR) + ) + } + + val processModuleInfoFile by tasks.registering(ProcessModuleInfoFile::class) { + moduleInfoFile.set(file("${target.name.ifEmpty { "." }}/src/module-info.java")) + processedModuleInfoFile.set(project.layout.buildDirectory.file("generated-sources/module-info-processor/module-info.java")) + } + + val compileJavaModuleInfo = tasks.register("compileModuleInfoJava", JavaCompile::class.java) { + val moduleName = project.name.replace('-', '.') // this module's name + val compileKotlinTask = + compilation.compileTaskProvider.get() as? org.jetbrains.kotlin.gradle.tasks.KotlinCompile + ?: error("Cannot access Kotlin compile task ${compilation.compileKotlinTaskName}") + val targetDir = compileKotlinTask.destinationDirectory.dir("../java9") + + // Use a Java 11 compiler for the module-info. + javaCompiler.set(javaToolchains.compilerFor { + languageVersion.set(JavaLanguageVersion.of(11)) + }) + + // Always compile kotlin classes before the module descriptor. + dependsOn(compileKotlinTask) + + // Add the module-info source file. + // Note that we use the parent dir and an include filter, + // this is needed for Gradle's module detection to work in + // org.gradle.api.tasks.compile.JavaCompile.createSpec + source(processModuleInfoFile.map { it.processedModuleInfoFile.asFile.get().parentFile }) + val generatedModuleInfoFile = processModuleInfoFile.flatMap { it.processedModuleInfoFile.asFile } + include { it.file == generatedModuleInfoFile.get() } + + // Set the task outputs and destination directory + outputs.dir(targetDir) + destinationDirectory.set(targetDir) + + // Configure JVM compatibility + sourceCompatibility = JavaVersion.VERSION_1_9.toString() + targetCompatibility = JavaVersion.VERSION_1_9.toString() + + // Set the Java release version. + options.release.set(9) + + // Ignore warnings about using 'requires transitive' on automatic modules. + // not needed when compiling with recent JDKs, e.g. 17 + options.compilerArgs.add("-Xlint:-requires-transitive-automatic") + + // Patch the compileKotlinJvm output classes into the compilation so exporting packages works correctly. + val destinationDirProperty = compileKotlinTask.destinationDirectory.asFile + options.compilerArgumentProviders.add { + val kotlinCompileDestinationDir = destinationDirProperty.get() + listOf("--patch-module", "$moduleName=$kotlinCompileDestinationDir") + } + + // Use the classpath of the compileKotlinJvm task. + // Also ensure that the module path is used instead of classpath. + classpath = compileKotlinTask.libraries + modularity.inferModulePath.set(true) + } + + tasks.named<Jar>(target.artifactsTaskName) { + manifest { + attributes("Multi-Release" to true) + } + from(compileJavaModuleInfo) { + // Include **only** file we are interested in as JavaCompile output also contains some tmp files + include("module-info.class") + into("META-INF/versions/9/") + } + } + } +} diff --git a/buildSrc/src/main/kotlin/Projects.kt b/buildSrc/src/main/kotlin/Projects.kt index af709893..2442c509 100644 --- a/buildSrc/src/main/kotlin/Projects.kt +++ b/buildSrc/src/main/kotlin/Projects.kt @@ -8,6 +8,7 @@ fun Project.version(target: String): String = property("${target}_version") as String val coreModule = "kotlinx-coroutines-core" +val jdk8ObsoleteModule = "kotlinx-coroutines-jdk8" val testModule = "kotlinx-coroutines-test" val multiplatform = setOf(coreModule, testModule) diff --git a/buildSrc/src/main/kotlin/animalsniffer-conventions.gradle.kts b/buildSrc/src/main/kotlin/animalsniffer-conventions.gradle.kts index f00a0b31..639245b6 100644 --- a/buildSrc/src/main/kotlin/animalsniffer-conventions.gradle.kts +++ b/buildSrc/src/main/kotlin/animalsniffer-conventions.gradle.kts @@ -17,6 +17,20 @@ configure(subprojects) { signature("net.sf.androidscents.signature:android-api-level-14:4.0_r4@signature") signature("org.codehaus.mojo.signature:java17:1.0@signature") } + + if (project.name == coreModule) { + // Specific files so nothing from core is accidentally skipped + tasks.withType<AnimalSniffer>().configureEach { + exclude("**/future/FutureKt*") + exclude("**/future/ContinuationHandler*") + exclude("**/future/CompletableFutureCoroutine*") + + exclude("**/stream/StreamKt*") + exclude("**/stream/StreamFlow*") + + exclude("**/time/TimeKt*") + } + } } } diff --git a/buildSrc/src/main/kotlin/configure-compilation-conventions.gradle.kts b/buildSrc/src/main/kotlin/configure-compilation-conventions.gradle.kts new file mode 100644 index 00000000..9e22b451 --- /dev/null +++ b/buildSrc/src/main/kotlin/configure-compilation-conventions.gradle.kts @@ -0,0 +1,31 @@ +/* + * Copyright 2016-2022 JetBrains s.r.o. Use of this source code is governed by the Apache 2.0 license. + */ + +import org.jetbrains.kotlin.gradle.dsl.KotlinCompile +import org.jetbrains.kotlin.gradle.dsl.KotlinCommonOptions + +configure(subprojects) { + val project = this + if (name in sourceless) return@configure + apply(plugin = "kotlinx-atomicfu") + tasks.withType<KotlinCompile<*>>().configureEach { + val isMainTaskName = name == "compileKotlin" || name == "compileKotlinJvm" + kotlinOptions { + languageVersion = getOverriddenKotlinLanguageVersion(project) + apiVersion = getOverriddenKotlinApiVersion(project) + if (isMainTaskName && versionsAreNotOverridden) { + allWarningsAsErrors = true + } + val newOptions = + listOf( + "-progressive", "-Xno-param-assertions", "-Xno-receiver-assertions", + "-Xno-call-assertions" + ) + optInAnnotations.map { "-opt-in=$it" } + freeCompilerArgs = freeCompilerArgs + newOptions + } + } +} + +val KotlinCommonOptions.versionsAreNotOverridden: Boolean + get() = languageVersion == null && apiVersion == null
\ No newline at end of file diff --git a/buildSrc/src/main/kotlin/java-modularity-conventions.gradle.kts b/buildSrc/src/main/kotlin/java-modularity-conventions.gradle.kts new file mode 100644 index 00000000..a5f72aa8 --- /dev/null +++ b/buildSrc/src/main/kotlin/java-modularity-conventions.gradle.kts @@ -0,0 +1,17 @@ +/* + * Copyright 2016-2021 JetBrains s.r.o. Use of this source code is governed by the Apache 2.0 license. + */ + +// Currently the compilation of the module-info fails for +// kotlinx-coroutines-play-services because it depends on Android JAR's +// which do not have an explicit module-info descriptor. +// Because the JAR's are all named `classes.jar`, +// the automatic module name also becomes `classes`. +// This conflicts since there are multiple JAR's with identical names. +val invalidModules = listOf("kotlinx-coroutines-play-services") + +configure(subprojects.filter { + !unpublished.contains(it.name) && !invalidModules.contains(it.name) && it.extensions.findByName("kotlin") != null +}) { + Java9Modularity.configure(project) +} diff --git a/buildSrc/src/main/kotlin/kover-conventions.gradle.kts b/buildSrc/src/main/kotlin/kover-conventions.gradle.kts index 052e2bb6..7a0a62f5 100644 --- a/buildSrc/src/main/kotlin/kover-conventions.gradle.kts +++ b/buildSrc/src/main/kotlin/kover-conventions.gradle.kts @@ -1,54 +1,77 @@ -import kotlinx.kover.api.* -import kotlinx.kover.tasks.* +import kotlinx.kover.gradle.plugin.dsl.* /* * Copyright 2016-2021 JetBrains s.r.o. Use of this source code is governed by the Apache 2.0 license. */ -apply(plugin = "kover") +plugins { + id("org.jetbrains.kotlinx.kover") +} val notCovered = sourceless + internal + unpublished val expectedCoverage = mutableMapOf( // These have lower coverage in general, it can be eventually fixed "kotlinx-coroutines-swing" to 70, // awaitFrame is not tested - "kotlinx-coroutines-javafx" to 39, // JavaFx is not tested on TC because its graphic subsystem cannot be initialized in headless mode + "kotlinx-coroutines-javafx" to 35, // JavaFx is not tested on TC because its graphic subsystem cannot be initialized in headless mode // Reactor has lower coverage in general due to various fatal error handling features - "kotlinx-coroutines-reactor" to 75) - -extensions.configure<KoverExtension> { - disabledProjects = notCovered - /* - * Is explicitly enabled on TC in a separate build step. - * Examples: - * ./gradlew :p:check -- doesn't verify coverage - * ./gradlew :p:check -Pkover.enabled=true -- verifies coverage - * ./gradlew :p:koverReport -Pkover.enabled=true -- generates report - */ - isDisabled = !(properties["kover.enabled"]?.toString()?.toBoolean() ?: false) - // TODO remove when updating Kover to version 0.5.x - intellijEngineVersion.set("1.0.657") -} + "kotlinx-coroutines-reactor" to 75 +) + +val conventionProject = project subprojects { val projectName = name if (projectName in notCovered) return@subprojects - tasks.withType<KoverVerificationTask> { - rule { - bound { - /* - * 85 is our baseline that we aim to raise to 90+. - * Missing coverage is typically due to bugs in the agent - * (e.g. signatures deprecated with an error are counted), - * sometimes it's various diagnostic `toString` or `catch` for OOMs/VerificationErrors, - * but some places are definitely worth visiting. - */ - minValue = expectedCoverage[projectName] ?: 85 // COVERED_LINES_PERCENTAGE + + project.apply(plugin = "org.jetbrains.kotlinx.kover") + conventionProject.dependencies.add("kover", this) + + extensions.configure<KoverProjectExtension>("kover") { + /* + * Is explicitly enabled on TC in a separate build step. + * Examples: + * ./gradlew :p:check -- doesn't verify coverage + * ./gradlew :p:check -Pkover.enabled=true -- verifies coverage + * ./gradlew :p:koverHtmlReport -Pkover.enabled=true -- generates HTML report + */ + if (properties["kover.enabled"]?.toString()?.toBoolean() != true) { + disable() + } + } + + extensions.configure<KoverReportExtension>("koverReport") { + defaults { + html { + setReportDir(conventionProject.layout.buildDirectory.dir("kover/${project.name}/html")) + } + + verify { + rule { + /* + * 85 is our baseline that we aim to raise to 90+. + * Missing coverage is typically due to bugs in the agent + * (e.g. signatures deprecated with an error are counted), + * sometimes it's various diagnostic `toString` or `catch` for OOMs/VerificationErrors, + * but some places are definitely worth visiting. + */ + minBound(expectedCoverage[projectName] ?: 85) // COVERED_LINES_PERCENTAGE + } } } } +} - tasks.withType<KoverHtmlReportTask> { - htmlReportDir.set(file(rootProject.buildDir.toString() + "/kover/" + project.name + "/html")) +koverReport { + defaults { + verify { + rule { + minBound(85) // COVERED_LINES_PERCENTAGE + } + } } } + +conventionProject.tasks.register("koverReport") { + dependsOn(conventionProject.tasks.named("koverHtmlReport")) +} diff --git a/bump-version.sh b/bump-version.sh index e49910f4..369c8866 100755 --- a/bump-version.sh +++ b/bump-version.sh @@ -1,35 +1,96 @@ -#!/bin/bash +#!/bin/sh -if [ "$#" -ne 2 ] +set -efu + +# the list of files that need to have the version updated in them +# +# limitations: +# * no newlines in names +# * no ' char in names +files=" +README.md +kotlinx-coroutines-core/README.md +kotlinx-coroutines-debug/README.md +kotlinx-coroutines-test/README.md +ui/coroutines-guide-ui.md +gradle.properties +integration-testing/gradle.properties +" + +# read gradle.properties to get the old version +set +e +old_version="$(git grep -hoP '(?<=^version=).*(?=-SNAPSHOT$)' gradle.properties)" +set -e +if [ "$?" -ne 0 ] then - echo "Use: ./bump-version old_version new_version" - exit + echo "Could not read the old version from gradle.properties." >&2 + if [ "$#" -ne 2 ] + then + echo "Please use this form instead: ./bump-version.sh old_version new_version" + exit 1 + fi fi -old_version=$1 -new_version=$2 +# check the command-line arguments for mentions of the version +if [ "$#" -eq 2 ] + then + echo "If you want to infer the version automatically, use the form: ./bump-version.sh new_version" >&2 + if [ -n "$old_version" -a "$1" != "$old_version" ] + then + echo "The provided old version ($1) is different from the one in gradle.properties ($old_version)." >&2 + echo "Proceeding anyway with the provided old version." >&2 + fi + old_version=$1 + new_version=$2 + elif [ "$#" -eq 1 ] + then + new_version=$1 + else + echo "Use: ./bump-version.sh new_version" >&2 + exit 1 +fi + + +# Escape dots, e.g. 1.0.0 -> 1\.0\.0 +escaped_old_version="$(printf "%s\n" "$old_version" | sed 's/[.]/\\./g')" update_version() { - echo "Updating version from '$old_version' to '$new_version' in $1" - sed -i.bak s/$old_version/$new_version/g $1 - rm $1.bak + file=$1 + to_undo=$2 + echo "Updating version from '$old_version' to '$new_version' in $1" >&2 + if [ -n "$(git diff --name-status -- "$file")" ] + then + printf "There are unstaged changes in '$file'. Refusing to proceed.\n" >&2 + [ -z "$to_undo" ] || eval "git checkout$to_undo" + exit 1 + fi + sed -i.bak "s/$escaped_old_version/$new_version/g" "$file" + rm -f "$1.bak" } -update_version "README.md" -update_version "kotlinx-coroutines-core/README.md" -update_version "kotlinx-coroutines-debug/README.md" -update_version "kotlinx-coroutines-test/README.md" -update_version "ui/coroutines-guide-ui.md" -update_version "gradle.properties" -update_version "integration-test/gradle.properties" +to_undo=$(printf "%s" "$files" | while read -r file; do + if [ -n "$file" ] + then + update_version "$file" "${to_undo:-}" + to_undo="${to_undo:-} '$file'" + echo -n " '$file'" + fi +done) -# Escape dots, e.g. 1.0.0 -> 1\.0\.0 -escaped_old_version=$(echo $old_version | sed s/[.]/\\\\./g) -result=$(find ./ -type f \( -iname \*.properties -o -iname \*.md \) | grep -v "\.gradle" | grep -v "build" | xargs -I{} grep -H "$escaped_old_version" {} | grep -v CHANGES.md | grep -v COMPATIBILITY.md) -if [ -z "$result" ]; +set +e +version_mentions=$( + find . -type f \( -iname '*.properties' -o -iname '*.md' \) \ + -not -iname CHANGES.md -not -iname CHANGES_UP_TO_1.7.md \ + -not -path ./integration/kotlinx-coroutines-jdk8/README.md \ + -exec git grep --fixed-strings --word "$old_version" {} + + ) +set -e +if [ -z "$version_mentions" ] then - echo "Done" + echo "Done. To undo, run this command:" >&2 + printf "git checkout%s\n" "$to_undo" >&2 else - echo "ERROR: Previous version is present in the project: $result" - exit -1 + echo "ERROR: Previous version is present in the project: $version_mentions" + [ -z "$to_undo" ] || eval "git checkout$to_undo" + exit 1 fi diff --git a/docs/cfg/buildprofiles.xml b/docs/cfg/buildprofiles.xml index ac4d04ad..d4a99434 100644 --- a/docs/cfg/buildprofiles.xml +++ b/docs/cfg/buildprofiles.xml @@ -3,7 +3,7 @@ <buildprofiles> <variables> <enable-browser-edits>true</enable-browser-edits> - <browser-edits-url>https://github.com/Kotlin/kotlinx.coroutines/edit/master/</browser-edits-url> + <browser-edits-url>https://github.com/Kotlin/kotlinx.coroutines/edit/master/docs/</browser-edits-url> <allow-indexable-eaps>true</allow-indexable-eaps> </variables> <build-profile product="kc"/> diff --git a/docs/images/after.png b/docs/images/after.png Binary files differindex 4ce15e8b..b1e138c6 100644 --- a/docs/images/after.png +++ b/docs/images/after.png diff --git a/docs/images/before.png b/docs/images/before.png Binary files differindex 31b91060..7386ee21 100644 --- a/docs/images/before.png +++ b/docs/images/before.png diff --git a/docs/images/coroutines-and-channels/aggregate.png b/docs/images/coroutines-and-channels/aggregate.png Binary files differnew file mode 100644 index 00000000..fed7b858 --- /dev/null +++ b/docs/images/coroutines-and-channels/aggregate.png diff --git a/docs/images/coroutines-and-channels/background.png b/docs/images/coroutines-and-channels/background.png Binary files differnew file mode 100644 index 00000000..0882eba3 --- /dev/null +++ b/docs/images/coroutines-and-channels/background.png diff --git a/docs/images/coroutines-and-channels/blocking.png b/docs/images/coroutines-and-channels/blocking.png Binary files differnew file mode 100644 index 00000000..6f04c1a7 --- /dev/null +++ b/docs/images/coroutines-and-channels/blocking.png diff --git a/docs/images/coroutines-and-channels/buffered-channel.png b/docs/images/coroutines-and-channels/buffered-channel.png Binary files differnew file mode 100644 index 00000000..009604b8 --- /dev/null +++ b/docs/images/coroutines-and-channels/buffered-channel.png diff --git a/docs/images/coroutines-and-channels/callbacks.png b/docs/images/coroutines-and-channels/callbacks.png Binary files differnew file mode 100644 index 00000000..c02d683c --- /dev/null +++ b/docs/images/coroutines-and-channels/callbacks.png diff --git a/docs/images/coroutines-and-channels/concurrency.png b/docs/images/coroutines-and-channels/concurrency.png Binary files differnew file mode 100644 index 00000000..e52e2eba --- /dev/null +++ b/docs/images/coroutines-and-channels/concurrency.png diff --git a/docs/images/coroutines-and-channels/conflated-channel.gif b/docs/images/coroutines-and-channels/conflated-channel.gif Binary files differnew file mode 100644 index 00000000..bf118b66 --- /dev/null +++ b/docs/images/coroutines-and-channels/conflated-channel.gif diff --git a/docs/images/coroutines-and-channels/generating-token.png b/docs/images/coroutines-and-channels/generating-token.png Binary files differnew file mode 100644 index 00000000..7c0aadc8 --- /dev/null +++ b/docs/images/coroutines-and-channels/generating-token.png diff --git a/docs/images/coroutines-and-channels/initial-window.png b/docs/images/coroutines-and-channels/initial-window.png Binary files differnew file mode 100644 index 00000000..5e7e3257 --- /dev/null +++ b/docs/images/coroutines-and-channels/initial-window.png diff --git a/docs/images/coroutines-and-channels/loading.gif b/docs/images/coroutines-and-channels/loading.gif Binary files differnew file mode 100644 index 00000000..8a862128 --- /dev/null +++ b/docs/images/coroutines-and-channels/loading.gif diff --git a/docs/images/coroutines-and-channels/progress-and-concurrency.png b/docs/images/coroutines-and-channels/progress-and-concurrency.png Binary files differnew file mode 100644 index 00000000..9d070c11 --- /dev/null +++ b/docs/images/coroutines-and-channels/progress-and-concurrency.png diff --git a/docs/images/coroutines-and-channels/progress.png b/docs/images/coroutines-and-channels/progress.png Binary files differnew file mode 100644 index 00000000..778dfb10 --- /dev/null +++ b/docs/images/coroutines-and-channels/progress.png diff --git a/docs/images/coroutines-and-channels/rendezvous-channel.png b/docs/images/coroutines-and-channels/rendezvous-channel.png Binary files differnew file mode 100644 index 00000000..23c7bc81 --- /dev/null +++ b/docs/images/coroutines-and-channels/rendezvous-channel.png diff --git a/docs/images/coroutines-and-channels/run-configuration.png b/docs/images/coroutines-and-channels/run-configuration.png Binary files differnew file mode 100644 index 00000000..9661f8f6 --- /dev/null +++ b/docs/images/coroutines-and-channels/run-configuration.png diff --git a/docs/images/coroutines-and-channels/suspend-requests.png b/docs/images/coroutines-and-channels/suspend-requests.png Binary files differnew file mode 100644 index 00000000..92f92f6b --- /dev/null +++ b/docs/images/coroutines-and-channels/suspend-requests.png diff --git a/docs/images/coroutines-and-channels/suspension-process.gif b/docs/images/coroutines-and-channels/suspension-process.gif Binary files differnew file mode 100644 index 00000000..f42dbe1e --- /dev/null +++ b/docs/images/coroutines-and-channels/suspension-process.gif diff --git a/docs/images/coroutines-and-channels/time-comparison.png b/docs/images/coroutines-and-channels/time-comparison.png Binary files differnew file mode 100644 index 00000000..b1a52a33 --- /dev/null +++ b/docs/images/coroutines-and-channels/time-comparison.png diff --git a/docs/images/coroutines-and-channels/unlimited-channel.png b/docs/images/coroutines-and-channels/unlimited-channel.png Binary files differnew file mode 100644 index 00000000..7cdef2a9 --- /dev/null +++ b/docs/images/coroutines-and-channels/unlimited-channel.png diff --git a/docs/images/coroutines-and-channels/using-channel-many-coroutines.png b/docs/images/coroutines-and-channels/using-channel-many-coroutines.png Binary files differnew file mode 100644 index 00000000..913a0ddc --- /dev/null +++ b/docs/images/coroutines-and-channels/using-channel-many-coroutines.png diff --git a/docs/images/coroutines-and-channels/using-channel.png b/docs/images/coroutines-and-channels/using-channel.png Binary files differnew file mode 100644 index 00000000..85094bd8 --- /dev/null +++ b/docs/images/coroutines-and-channels/using-channel.png diff --git a/docs/images/variable-optimised-out.png b/docs/images/variable-optimised-out.png Binary files differnew file mode 100644 index 00000000..2db471e9 --- /dev/null +++ b/docs/images/variable-optimised-out.png diff --git a/docs/kc.tree b/docs/kc.tree index 82877f44..9fa1e113 100644 --- a/docs/kc.tree +++ b/docs/kc.tree @@ -10,7 +10,7 @@ <chunk include-id="coroutines"> <toc-element id="coroutines-guide.md"/> <toc-element id="coroutines-basics.md" accepts-web-file-names="basics.html,coroutines-basic-jvm.html"/> - <toc-element toc-title="Intro to coroutines and channels – hands-on tutorial" href="https://play.kotlinlang.org/hands-on/Introduction%20to%20Coroutines%20and%20Channels/"/> + <toc-element id="coroutines-and-channels.md"/> <toc-element id="cancellation-and-timeouts.md"/> <toc-element id="composing-suspending-functions.md"/> <toc-element id="coroutine-context-and-dispatchers.md"/> diff --git a/docs/topics/cancellation-and-timeouts.md b/docs/topics/cancellation-and-timeouts.md index 0cbafb59..c574e126 100644 --- a/docs/topics/cancellation-and-timeouts.md +++ b/docs/topics/cancellation-and-timeouts.md @@ -379,10 +379,11 @@ The timeout event in [withTimeout] is asynchronous with respect to the code runn even right before the return from inside of the timeout block. Keep this in mind if you open or acquire some resource inside the block that needs closing or release outside of the block. -For example, here we imitate a closeable resource with the `Resource` class, that simply keeps track of how many times -it was created by incrementing the `acquired` counter and decrementing this counter from its `close` function. -Let us run a lot of coroutines with the small timeout try acquire this resource from inside -of the `withTimeout` block after a bit of delay and release it from outside. +For example, here we imitate a closeable resource with the `Resource` class that simply keeps track of how many times +it was created by incrementing the `acquired` counter and decrementing the counter in its `close` function. +Now let us create a lot of coroutines, each of which creates a `Resource` at the end of the `withTimeout` block +and releases the resource outside the block. We add a small delay so that it is more likely that the timeout occurs +right when the `withTimeout` block is already finished, which will cause a resource leak. ```kotlin import kotlinx.coroutines.* @@ -397,7 +398,7 @@ class Resource { fun main() { runBlocking { - repeat(100_000) { // Launch 100K coroutines + repeat(10_000) { // Launch 10K coroutines launch { val resource = withTimeout(60) { // Timeout of 60 ms delay(50) // Delay for 50 ms @@ -420,16 +421,16 @@ fun main() { <!--- CLEAR --> -If you run the above code you'll see that it does not always print zero, though it may depend on the timings -of your machine you may need to tweak timeouts in this example to actually see non-zero values. +If you run the above code, you'll see that it does not always print zero, though it may depend on the timings +of your machine. You may need to tweak the timeout in this example to actually see non-zero values. -> Note that incrementing and decrementing `acquired` counter here from 100K coroutines is completely safe, -> since it always happens from the same main thread. More on that will be explained in the chapter -> on coroutine context. +> Note that incrementing and decrementing `acquired` counter here from 10K coroutines is completely thread-safe, +> since it always happens from the same thread, the one used by `runBlocking`. +> More on that will be explained in the chapter on coroutine context. > {type="note"} -To work around this problem you can store a reference to the resource in the variable as opposed to returning it +To work around this problem you can store a reference to the resource in a variable instead of returning it from the `withTimeout` block. ```kotlin @@ -445,7 +446,7 @@ class Resource { fun main() { //sampleStart runBlocking { - repeat(100_000) { // Launch 100K coroutines + repeat(10_000) { // Launch 10K coroutines launch { var resource: Resource? = null // Not acquired yet try { diff --git a/docs/topics/coroutines-and-channels.md b/docs/topics/coroutines-and-channels.md new file mode 100644 index 00000000..b60d8982 --- /dev/null +++ b/docs/topics/coroutines-and-channels.md @@ -0,0 +1,1551 @@ +[//]: # (title: Coroutines and channels − tutorial) + +In this tutorial, you'll learn how to use coroutines in IntelliJ IDEA to perform network requests without blocking the +underlying thread or callbacks. + +> No prior knowledge of coroutines is required, but you're expected to be familiar with basic Kotlin syntax. +> +{type="tip"} + +You'll learn: + +* Why and how to use suspending functions to perform network requests. +* How to send requests concurrently using coroutines. +* How to share information between different coroutines using channels. + +For network requests, you'll need the [Retrofit](https://square.github.io/retrofit/) library, but the approach shown in +this tutorial works similarly for any other libraries that support coroutines. + +> You can find solutions for all of the tasks on the `solutions` branch of the [project's repository](http://github.com/kotlin-hands-on/intro-coroutines). +> +{type="tip"} + +## Before you start + +1. Download and install the latest version of [IntelliJ IDEA](https://www.jetbrains.com/idea/download/index.html). +2. Clone the [project template](http://github.com/kotlin-hands-on/intro-coroutines) by choosing **Get from VCS** on the + Welcome screen or selecting **File | New | Project from Version Control**. + + You can also clone it from the command line: + + ```Bash + git clone https://github.com/kotlin-hands-on/intro-coroutines + ``` + +### Generate a GitHub developer token + +You'll be using the GitHub API in your project. To get access, provide your GitHub account name and either a password or a +token. If you have two-factor authentication enabled, a token will be enough. + +Generate a new GitHub token to use the GitHub API with [your account](https://github.com/settings/tokens/new): + +1. Specify the name of your token, for example, `coroutines-tutorial`: + + {width=700} + +2. Do not select any scopes. Click **Generate token** at the bottom of the page. +3. Copy the generated token. + +### Run the code + +The program loads the contributors for all of the repositories under the given organization (named “kotlin” by default). +Later you'll add logic to sort the users by the number of their contributions. + +1. Open the `src/contributors/main.kt` file and run the `main()` function. You'll see the following window: + + {width=500} + + If the font is too small, adjust it by changing the value of `setDefaultFontSize(18f)` in the `main()` function. + +2. Provide your GitHub username and token (or password) in the corresponding fields. +3. Make sure that the _BLOCKING_ option is selected in the _Variant_ dropdown menu. +4. Click _Load contributors_. The UI should freeze for some time and then show the list of contributors. +5. Open the program output to ensure the data has been loaded. The list of contributors is logged after each successful request. + +There are different ways of implementing this logic: by using [blocking requests](#blocking-requests) +or [callbacks](#callbacks). You'll compare these solutions with one that uses [coroutines](#coroutines) and see how +[channels](#channels) can be used to share information between different coroutines. + +## Blocking requests + +You will use the [Retrofit](https://square.github.io/retrofit/) library to perform HTTP requests to GitHub. It allows +requesting the list of repositories under the given organization and the list of contributors for each repository: + +```kotlin +interface GitHubService { + @GET("orgs/{org}/repos?per_page=100") + fun getOrgReposCall( + @Path("org") org: String + ): Call<List<Repo>> + + @GET("repos/{owner}/{repo}/contributors?per_page=100") + fun getRepoContributorsCall( + @Path("owner") owner: String, + @Path("repo") repo: String + ): Call<List<User>> +} +``` + +This API is used by the `loadContributorsBlocking()` function to fetch the list of contributors for the given organization. + +1. Open `src/tasks/Request1Blocking.kt` to see its implementation: + + ```kotlin + fun loadContributorsBlocking(service: GitHubService, req: RequestData): List<User> { + val repos = service + .getOrgReposCall(req.org) // #1 + .execute() // #2 + .also { logRepos(req, it) } // #3 + .body() ?: emptyList() // #4 + + return repos.flatMap { repo -> + service + .getRepoContributorsCall(req.org, repo.name) // #1 + .execute() // #2 + .also { logUsers(repo, it) } // #3 + .bodyList() // #4 + }.aggregate() + } + ``` + + * At first, you get a list of the repositories under the given organization and store it in the `repos` list. Then for + each repository, the list of contributors is requested, and all of the lists are merged into one final list of + contributors. + * `getOrgReposCall()` and `getRepoContributorsCall()` both return an instance of the `*Call` class (`#1`). At this point, + no request is sent. + * `*Call.execute()` is then invoked to perform the request (`#2`). `execute()` is a synchronous call that blocks the + underlying thread. + * When you get the response, the result is logged by calling the specific `logRepos()` and `logUsers()` functions (`#3`). + If the HTTP response contains an error, this error will be logged here. + * Finally, get the response's body, which contains the data you need. For this tutorial, you'll use an empty list as a + result in case there is an error, and you'll log the corresponding error (`#4`). + +2. To avoid repeating `.body() ?: emptyList()`, an extension function `bodyList()` is declared: + + ```kotlin + fun <T> Response<List<T>>.bodyList(): List<T> { + return body() ?: emptyList() + } + ``` + +3. Run the program again and take a look at the system output in IntelliJ IDEA. It should have something like this: + + ```text + 1770 [AWT-EventQueue-0] INFO Contributors - kotlin: loaded 40 repos + 2025 [AWT-EventQueue-0] INFO Contributors - kotlin-examples: loaded 23 contributors + 2229 [AWT-EventQueue-0] INFO Contributors - kotlin-koans: loaded 45 contributors + ... + ``` + + * The first item on each line is the number of milliseconds that have passed since the program started, then the thread + name in square brackets. You can see from which thread the loading request is called. + * The final item on each line is the actual message: how many repositories or contributors were loaded. + + This log output demonstrates that all of the results were logged from the main thread. When you run the code with a _BLOCKING_ + option, the window freezes and doesn't react to input until the loading is finished. All of the requests are executed from + the same thread as the one called `loadContributorsBlocking()` is from, which is the main UI thread (in Swing, it's an AWT + event dispatching thread). This main thread becomes blocked, and that's why the UI is frozen: + + {width=700} + + After the list of contributors has loaded, the result is updated. + +4. In `src/contributors/Contributors.kt`, find the `loadContributors()` function responsible for choosing how + the contributors are loaded and look at how `loadContributorsBlocking()` is called: + + ```kotlin + when (getSelectedVariant()) { + BLOCKING -> { // Blocking UI thread + val users = loadContributorsBlocking(service, req) + updateResults(users, startTime) + } + } + ``` + + * The `updateResults()` call goes right after the `loadContributorsBlocking()` call. + * `updateResults()` updates the UI, so it must always be called from the UI thread. + * Since `loadContributorsBlocking()` is also called from the UI thread, the UI thread becomes blocked and the UI is + frozen. + +### Task 1 + +The first task helps you familiarize yourself with the task domain. Currently, each contributor's name is repeated +several times, once for every project they have taken part in. Implement the `aggregate()` function combining the users +so that each contributor is added only once. The `User.contributions` property should contain the total number of +contributions of the given user to _all_ the projects. The resulting list should be sorted in descending order according +to the number of contributions. + +Open `src/tasks/Aggregation.kt` and implement the `List<User>.aggregate()` function. Users should be sorted by the total +number of their contributions. + +The corresponding test file `test/tasks/AggregationKtTest.kt` shows an example of the expected result. + +> You can jump between the source code and the test class automatically by using the [IntelliJ IDEA shortcut](https://www.jetbrains.com/help/idea/create-tests.html#test-code-navigation) +> `Ctrl+Shift+T` / `⇧ ⌘ T`. +> +{type="tip"} + +After implementing this task, the resulting list for the "kotlin" organization should be similar to the following: + +{width=500} + +#### Solution for task 1 {initial-collapse-state="collapsed"} + +1. To group users by login, use [`groupBy()`](https://kotlinlang.org/api/latest/jvm/stdlib/kotlin.collections/group-by.html), + which returns a map from a login to all occurrences of the user with this login in different repositories. +2. For each map entry, count the total number of contributions for each user and create a new instance of the `User` class + by the given name and total of contributions. +3. Sort the resulting list in descending order: + + ```kotlin + fun List<User>.aggregate(): List<User> = + groupBy { it.login } + .map { (login, group) -> User(login, group.sumOf { it.contributions }) } + .sortedByDescending { it.contributions } + ``` + +An alternative solution is to use the [`groupingBy()`](https://kotlinlang.org/api/latest/jvm/stdlib/kotlin.collections/grouping-by.html) +function instead of `groupBy()`. + +## Callbacks + +The previous solution works, but it blocks the thread and therefore freezes the UI. A traditional approach that avoids this +is to use _callbacks_. + +Instead of calling the code that should be invoked right after the operation is completed, you can extract it +into a separate callback, often a lambda, and pass that lambda to the caller in order for it to be called later. + +To make the UI responsive, you can either move the whole computation to a separate thread or switch to the Retrofit API +which uses callbacks instead of blocking calls. + +### Use a background thread + +1. Open `src/tasks/Request2Background.kt` and see its implementation. First, the whole computation is moved to a different + thread. The `thread()` function starts a new thread: + + ```kotlin + thread { + loadContributorsBlocking(service, req) + } + ``` + + Now that all of the loading has been moved to a separate thread, the main thread is free and can be occupied by other + tasks: + + {width=700} + +2. The signature of the `loadContributorsBackground()` function changes. It takes an `updateResults()` + callback as the last argument to call it after all the loading completes: + + ```kotlin + fun loadContributorsBackground( + service: GitHubService, req: RequestData, + updateResults: (List<User>) -> Unit + ) + ``` + +3. Now when the `loadContributorsBackground()` is called, the `updateResults()` call goes in the callback, not immediately + afterward as it did before: + + ```kotlin + loadContributorsBackground(service, req) { users -> + SwingUtilities.invokeLater { + updateResults(users, startTime) + } + } + ``` + + By calling `SwingUtilities.invokeLater`, you ensure that the `updateResults()` call, which updates the results, + happens on the main UI thread (AWT event dispatching thread). + +However, if you try to load the contributors via the `BACKGROUND` option, you can see that the list is updated but +nothing changes. + +### Task 2 + +Fix the `loadContributorsBackground()` function in `src/tasks/Request2Background.kt` so that the resulting list is shown +in the UI. + +#### Solution for task 2 {initial-collapse-state="collapsed"} + +If you try to load the contributors, you can see in the log that the contributors are loaded but the result isn't displayed. +To fix this, call `updateResults()` on the resulting list of users: + +```kotlin +thread { + updateResults(loadContributorsBlocking(service, req)) +} +``` + +Make sure to call the logic passed in the callback explicitly. Otherwise, nothing will happen. + +### Use the Retrofit callback API + +In the previous solution, the whole loading logic is moved to the background thread, but that still isn't the best use of +resources. All of the loading requests go sequentially and the thread is blocked while waiting for the loading result, +while it could have been occupied by other tasks. Specifically, the thread could start loading another request to +receive the entire result earlier. + +Handling the data for each repository should then be divided into two parts: loading and processing the +resulting response. The second _processing_ part should be extracted into a callback. + +The loading for each repository can then be started before the result for the previous repository is received (and the +corresponding callback is called): + +{width=700} + +The Retrofit callback API can help achieve this. The `Call.enqueue()` function starts an HTTP request and takes a +callback as an argument. In this callback, you need to specify what needs to be done after each request. + +Open `src/tasks/Request3Callbacks.kt` and see the implementation of `loadContributorsCallbacks()` that uses this API: + +```kotlin +fun loadContributorsCallbacks( + service: GitHubService, req: RequestData, + updateResults: (List<User>) -> Unit +) { + service.getOrgReposCall(req.org).onResponse { responseRepos -> // #1 + logRepos(req, responseRepos) + val repos = responseRepos.bodyList() + + val allUsers = mutableListOf<User>() + for (repo in repos) { + service.getRepoContributorsCall(req.org, repo.name) + .onResponse { responseUsers -> // #2 + logUsers(repo, responseUsers) + val users = responseUsers.bodyList() + allUsers += users + } + } + } + // TODO: Why doesn't this code work? How to fix that? + updateResults(allUsers.aggregate()) + } +``` + +* For convenience, this code fragment uses the `onResponse()` extension function declared in the same file. It takes a + lambda as an argument rather than an object expression. +* The logic for handling the responses is extracted into callbacks: the corresponding lambdas start at lines `#1` and `#2`. + +However, the provided solution doesn't work. If you run the program and load contributors by choosing the _CALLBACKS_ +option, you'll see that nothing is shown. However, the tests that immediately return the result pass. + +Think about why the given code doesn't work as expected and try to fix it, or see the solutions below. + +### Task 3 (optional) + +Rewrite the code in the `src/tasks/Request3Callbacks.kt` file so that the loaded list of contributors is shown. + +#### The first attempted solution for task 3 {initial-collapse-state="collapsed"} + +In the current solution, many requests are started concurrently, which decreases the total loading time. However, +the result isn't loaded. This is because the `updateResults()` callback is called right after all of the loading requests are started, +before the `allUsers` list has been filled with the data. + +You could try to fix this with a change like the following: + +```kotlin +val allUsers = mutableListOf<User>() +for ((index, repo) in repos.withIndex()) { // #1 + service.getRepoContributorsCall(req.org, repo.name) + .onResponse { responseUsers -> + logUsers(repo, responseUsers) + val users = responseUsers.bodyList() + allUsers += users + if (index == repos.lastIndex) { // #2 + updateResults(allUsers.aggregate()) + } + } +} +``` + +* First, you iterate over the list of repos with an index (`#1`). +* Then, from each callback, you check whether it's the last iteration (`#2`). +* And if that's the case, the result is updated. + +However, this code also fails to achieve our objective. Try to find the answer yourself, or see the solution below. + +#### The second attempted solution for task 3 {initial-collapse-state="collapsed"} + +Since the loading requests are started concurrently, there's no guarantee that the result for the last one comes last. The +results can come in any order. + +Thus, if you compare the current index with the `lastIndex` as a condition for completion, you risk losing the results for +some repos. + +If the request that processes the last repo returns faster than some prior requests (which is likely to happen), all of the +results for requests that take more time will be lost. + +One way to fix this is to introduce an index and check whether all of the repositories have already been processed: + +```kotlin +val allUsers = Collections.synchronizedList(mutableListOf<User>()) +val numberOfProcessed = AtomicInteger() +for (repo in repos) { + service.getRepoContributorsCall(req.org, repo.name) + .onResponse { responseUsers -> + logUsers(repo, responseUsers) + val users = responseUsers.bodyList() + allUsers += users + if (numberOfProcessed.incrementAndGet() == repos.size) { + updateResults(allUsers.aggregate()) + } + } +} +``` + +This code uses a synchronized version of the list and `AtomicInteger()` because, in general, there's no guarantee that +different callbacks that process `getRepoContributors()` requests will always be called from the same thread. + +#### The third attempted solution for task 3 {initial-collapse-state="collapsed"} + +An even better solution is to use the `CountDownLatch` class. It stores a counter initialized with the number of +repositories. This counter is decremented after processing each repository. It then waits until the latch is counted +down to zero before updating the results: + +```kotlin +val countDownLatch = CountDownLatch(repos.size) +for (repo in repos) { + service.getRepoContributorsCall(req.org, repo.name) + .onResponse { responseUsers -> + // processing repository + countDownLatch.countDown() + } +} +countDownLatch.await() +updateResults(allUsers.aggregate()) +``` + +The result is then updated from the main thread. This is more direct than delegating the logic to the child threads. + +After reviewing these three attempts at a solution, you can see that writing correct code with callbacks is non-trivial +and error-prone, especially when several underlying threads and synchronization occur. + +> As an additional exercise, you can implement the same logic using a reactive approach with the RxJava library. All of the +> necessary dependencies and solutions for using RxJava can be found in a separate `rx` branch. It is also possible to +> complete this tutorial and implement or check the proposed Rx versions for a proper comparison. +> +{type="tip"} + +## Suspending functions + +You can implement the same logic using suspending functions. Instead of returning `Call<List<Repo>>`, define the API +call as a [suspending function](composing-suspending-functions.md) as follows: + +```kotlin +interface GitHubService { + @GET("orgs/{org}/repos?per_page=100") + suspend fun getOrgRepos( + @Path("org") org: String + ): List<Repo> +} +``` + +* `getOrgRepos()` is defined as a `suspend` function. When you use a suspending function to perform a request, the + underlying thread isn't blocked. More details about how this works will come in later sections. +* `getOrgRepos()` returns the result directly instead of returning a `Call`. If the result is unsuccessful, an + exception is thrown. + +Alternatively, Retrofit allows returning the result wrapped in `Response`. In this case, the result body is +provided, and it is possible to check for errors manually. This tutorial uses the versions that return `Response`. + +In `src/contributors/GitHubService.kt`, add the following declarations to the `GitHubService` interface: + +```kotlin +interface GitHubService { + // getOrgReposCall & getRepoContributorsCall declarations + + @GET("orgs/{org}/repos?per_page=100") + suspend fun getOrgRepos( + @Path("org") org: String + ): Response<List<Repo>> + + @GET("repos/{owner}/{repo}/contributors?per_page=100") + suspend fun getRepoContributors( + @Path("owner") owner: String, + @Path("repo") repo: String + ): Response<List<User>> +} +``` + +### Task 4 + +Your task is to change the code of the function that loads contributors to make use of two new suspending functions, +`getOrgRepos()` and `getRepoContributors()`. The new `loadContributorsSuspend()` function is marked as `suspend` to use the +new API. + +> Suspending functions can't be called everywhere. Calling a suspending function from `loadContributorsBlocking()` will +> result in an error with the message "Suspend function 'getOrgRepos' should be called only from a coroutine or another +> suspend function". +> +{type="note"} + +1. Copy the implementation of `loadContributorsBlocking()` that is defined in `src/tasks/Request1Blocking.kt` + into the `loadContributorsSuspend()` that is defined in `src/tasks/Request4Suspend.kt`. +2. Modify the code so that the new suspending functions are used instead of the ones that return `Call`s. +3. Run the program by choosing the _SUSPEND_ option and ensure that the UI is still responsive while the GitHub requests + are performed. + +#### Solution for task 4 {initial-collapse-state="collapsed"} + +Replace `.getOrgReposCall(req.org).execute()` with `.getOrgRepos(req.org)` and repeat the same replacement for the +second "contributors" request: + +```kotlin +suspend fun loadContributorsSuspend(service: GitHubService, req: RequestData): List<User> { + val repos = service + .getOrgRepos(req.org) + .also { logRepos(req, it) } + .bodyList() + + return repos.flatMap { repo -> + service.getRepoContributors(req.org, repo.name) + .also { logUsers(repo, it) } + .bodyList() + }.aggregate() +} +``` + +* `loadContributorsSuspend()` should be defined as a `suspend` function. +* You no longer need to call `execute`, which returned the `Response` before, because now the API functions return + the `Response` directly. Note that this detail is specific to the Retrofit library. With other libraries, the API will be different, + but the concept is the same. + +## Coroutines + +The code with suspending functions looks similar to the "blocking" version. The major difference from the blocking version +is that instead of blocking the thread, the coroutine is suspended: + +```text +block -> suspend +thread -> coroutine +``` + +> Coroutines are often called lightweight threads because you can run code on coroutines, similar to how you run code on +> threads. The operations that were blocking before (and had to be avoided) can now suspend the coroutine instead. +> +{type="note"} + +### Starting a new coroutine + +If you look at how `loadContributorsSuspend()` is used in `src/contributors/Contributors.kt`, you can see that it's +called inside `launch`. `launch` is a library function that takes a lambda as an argument: + +```kotlin +launch { + val users = loadContributorsSuspend(req) + updateResults(users, startTime) +} +``` + +Here `launch` starts a new computation that is responsible for loading the data and showing the results. The computation +is suspendable – when performing network requests, it is suspended and releases the underlying thread. +When the network request returns the result, the computation is resumed. + +Such a suspendable computation is called a _coroutine_. So, in this case, `launch` _starts a new coroutine_ responsible +for loading data and showing the results. + +Coroutines run on top of threads and can be suspended. When a coroutine is suspended, the +corresponding computation is paused, removed from the thread, and stored in memory. Meanwhile, the thread is free to be +occupied by other tasks: + +{width=700} + +When the computation is ready to be continued, it is returned to a thread (not necessarily the same one). + +In the `loadContributorsSuspend()` example, each "contributors" request now waits for the result using the suspension +mechanism. First, the new request is sent. Then, while waiting for the response, the whole "load contributors" coroutine +that was started by the `launch` function is suspended. + +The coroutine resumes only after the corresponding response is received: + +{width=700} + +While the response is waiting to be received, the thread is free to be occupied by other tasks. The UI stays responsive, +despite all the requests taking place on the main UI thread: + +1. Run the program using the _SUSPEND_ option. The log confirms that all of the requests are sent to the main UI thread: + + ```text + 2538 [AWT-EventQueue-0 @coroutine#1] INFO Contributors - kotlin: loaded 30 repos + 2729 [AWT-EventQueue-0 @coroutine#1] INFO Contributors - ts2kt: loaded 11 contributors + 3029 [AWT-EventQueue-0 @coroutine#1] INFO Contributors - kotlin-koans: loaded 45 contributors + ... + 11252 [AWT-EventQueue-0 @coroutine#1] INFO Contributors - kotlin-coroutines-workshop: loaded 1 contributors + ``` + +2. The log can show you which coroutine the corresponding code is running on. To enable it, open **Run | Edit configurations** + and add the `-Dkotlinx.coroutines.debug` VM option: + + {width=500} + + The coroutine name will be attached to the thread name while `main()` is run with this option. You can also + modify the template for running all of the Kotlin files and enable this option by default. + +Now all of the code runs on one coroutine, the "load contributors" coroutine mentioned above, denoted as `@coroutine#1`. +While waiting for the result, you shouldn't reuse the thread for sending other requests because the code is +written sequentially. The new request is sent only when the previous result is received. + +Suspending functions treat the thread fairly and don't block it for "waiting". However, this doesn't yet bring any concurrency +into the picture. + +## Concurrency + +Kotlin coroutines are much less resource-intensive than threads. +Each time you want to start a new computation asynchronously, you can create a new coroutine instead. + +To start a new coroutine, use one of the main _coroutine builders_: `launch`, `async`, or `runBlocking`. Different +libraries can define additional coroutine builders. + +`async` starts a new coroutine and returns a `Deferred` object. `Deferred` represents a concept known by other names +such as `Future` or `Promise`. It stores a computation, but it _defers_ the moment you get the final result; +it _promises_ the result sometime in the _future_. + +The main difference between `async` and `launch` is that `launch` is used to start a computation that isn't expected to +return a specific result. `launch` returns a `Job` that represents the coroutine. It is possible to wait until it completes +by calling `Job.join()`. + +`Deferred` is a generic type that extends `Job`. An `async` call can return a `Deferred<Int>` or a `Deferred<CustomType>`, +depending on what the lambda returns (the last expression inside the lambda is the result). + +To get the result of a coroutine, you can call `await()` on the `Deferred` instance. While waiting for the result, +the coroutine that this `await()` is called from is suspended: + +```kotlin +import kotlinx.coroutines.* + +fun main() = runBlocking { + val deferred: Deferred<Int> = async { + loadData() + } + println("waiting...") + println(deferred.await()) +} + +suspend fun loadData(): Int { + println("loading...") + delay(1000L) + println("loaded!") + return 42 +} +``` + +`runBlocking` is used as a bridge between regular and suspending functions, or between the blocking and non-blocking worlds. It works +as an adaptor for starting the top-level main coroutine. It is intended primarily to be used in `main()` functions and +tests. + +> Watch [this video](https://www.youtube.com/watch?v=zEZc5AmHQhk) for a better understanding of coroutines. +> +{type="tip"} + +If there is a list of deferred objects, you can call `awaitAll()` to await the results of all of them: + +```kotlin +import kotlinx.coroutines.* + +fun main() = runBlocking { + val deferreds: List<Deferred<Int>> = (1..3).map { + async { + delay(1000L * it) + println("Loading $it") + it + } + } + val sum = deferreds.awaitAll().sum() + println("$sum") +} +``` + +When each "contributors" request is started in a new coroutine, all of the requests are started asynchronously. A new request +can be sent before the result for the previous one is received: + +{width=700} + +The total loading time is approximately the same as in the _CALLBACKS_ version, but it doesn't need any callbacks. +What's more, `async` explicitly emphasizes which parts run concurrently in the code. + +### Task 5 + +In the `Request5Concurrent.kt` file, implement a `loadContributorsConcurrent()` function by using the +previous `loadContributorsSuspend()` function. + +#### Tip for task 5 {initial-collapse-state="collapsed"} + +You can only start a new coroutine inside a coroutine scope. Copy the content +from `loadContributorsSuspend()` to the `coroutineScope` call so that you can call `async` functions there: + +```kotlin +suspend fun loadContributorsConcurrent( + service: GitHubService, + req: RequestData +): List<User> = coroutineScope { + // ... +} +``` + +Base your solution on the following scheme: + +```kotlin +val deferreds: List<Deferred<List<User>>> = repos.map { repo -> + async { + // load contributors for each repo + } +} +deferreds.awaitAll() // List<List<User>> +``` + +#### Solution for task 5 {initial-collapse-state="collapsed"} + +Wrap each "contributors" request with `async` to create as many coroutines as there are repositories. `async` +returns `Deferred<List<User>>`. This is not an issue because creating new coroutines is not very resource-intensive, so you can +create as many as you need. + +1. You can no longer use `flatMap` because the `map` result is now a list of `Deferred` objects, not a list of lists. + `awaitAll()` returns `List<List<User>>`, so call `flatten().aggregate()` to get the result: + + ```kotlin + suspend fun loadContributorsConcurrent( + service: GitHubService, + req: RequestData + ): List<User> = coroutineScope { + val repos = service + .getOrgRepos(req.org) + .also { logRepos(req, it) } + .bodyList() + + val deferreds: List<Deferred<List<User>>> = repos.map { repo -> + async { + service.getRepoContributors(req.org, repo.name) + .also { logUsers(repo, it) } + .bodyList() + } + } + deferreds.awaitAll().flatten().aggregate() + } + ``` + +2. Run the code and check the log. All of the coroutines still run on the main UI thread because + multithreading hasn't been employed yet, but you can already see the benefits of running coroutines concurrently. +3. To change this code to run "contributors" coroutines on different threads from the common thread pool, + specify `Dispatchers.Default` as the context argument for the `async` function: + + ```kotlin + async(Dispatchers.Default) { } + ``` + + * `CoroutineDispatcher` determines what thread or threads the corresponding coroutine should be run on. If you don't + specify one as an argument, `async` will use the dispatcher from the outer scope. + * `Dispatchers.Default` represents a shared pool of threads on the JVM. This pool provides a means for parallel execution. + It consists of as many threads as there are CPU cores available, but it will still have two threads if there's only one core. + +4. Modify the code in the `loadContributorsConcurrent()` function to start new coroutines on different threads from the + common thread pool. Also, add additional logging before sending the request: + + ```kotlin + async(Dispatchers.Default) { + log("starting loading for ${repo.name}") + service.getRepoContributors(req.org, repo.name) + .also { logUsers(repo, it) } + .bodyList() + } + ``` + +5. Run the program once again. In the log, you can see that each coroutine can be started on one thread from the + thread pool and resumed on another: + + ```text + 1946 [DefaultDispatcher-worker-2 @coroutine#4] INFO Contributors - starting loading for kotlin-koans + 1946 [DefaultDispatcher-worker-3 @coroutine#5] INFO Contributors - starting loading for dokka + 1946 [DefaultDispatcher-worker-1 @coroutine#3] INFO Contributors - starting loading for ts2kt + ... + 2178 [DefaultDispatcher-worker-1 @coroutine#4] INFO Contributors - kotlin-koans: loaded 45 contributors + 2569 [DefaultDispatcher-worker-1 @coroutine#5] INFO Contributors - dokka: loaded 36 contributors + 2821 [DefaultDispatcher-worker-2 @coroutine#3] INFO Contributors - ts2kt: loaded 11 contributors + ``` + + For instance, in this log excerpt, `coroutine#4` is started on the `worker-2` thread and continued on the + `worker-1` thread. + +In `src/contributors/Contributors.kt`, check the implementation of the _CONCURRENT_ option: + +1. To run the coroutine only on the main UI thread, specify `Dispatchers.Main` as an argument: + + ```kotlin + launch(Dispatchers.Main) { + updateResults() + } + ``` + + * If the main thread is busy when you start a new coroutine on it, + the coroutine becomes suspended and scheduled for execution on this thread. The coroutine will only resume when the + thread becomes free. + * It's considered good practice to use the dispatcher from the outer scope rather than explicitly specifying it on each + end-point. If you define `loadContributorsConcurrent()` without passing `Dispatchers.Default` as an + argument, you can call this function in any context: with a `Default` dispatcher, with + the main UI thread, or with a custom dispatcher. + * As you'll see later, when calling `loadContributorsConcurrent()` from tests, you can call it in the context + with `TestDispatcher`, which simplifies testing. That makes this solution much more flexible. + +2. To specify the dispatcher on the caller side, apply the following change to the project while + letting `loadContributorsConcurrent` start coroutines in the inherited context: + + ```kotlin + launch(Dispatchers.Default) { + val users = loadContributorsConcurrent(service, req) + withContext(Dispatchers.Main) { + updateResults(users, startTime) + } + } + ``` + + * `updateResults()` should be called on the main UI thread, so you call it with the context of `Dispatchers.Main`. + * `withContext()` calls the given code with the specified coroutine context, is suspended until it completes, and returns + the result. An alternative but more verbose way to express this would be to start a new coroutine and explicitly + wait (by suspending) until it completes: `launch(context) { ... }.join()`. + +3. Run the code and ensure that the coroutines are executed on the threads from the thread pool. + +## Structured concurrency + +* The _coroutine scope_ is responsible for the structure and parent-child relationships between different coroutines. New + coroutines usually need to be started inside a scope. +* The _coroutine context_ stores additional technical information used to run a given coroutine, like the coroutine custom + name, or the dispatcher specifying the threads the coroutine should be scheduled on. + +When `launch`, `async`, or `runBlocking` are used to start a new coroutine, they automatically create the corresponding +scope. All of these functions take a lambda with a receiver as an argument, and `CoroutineScope` is the implicit receiver type: + +```kotlin +launch { /* this: CoroutineScope */ } +``` + +* New coroutines can only be started inside a scope. +* `launch` and `async` are declared as extensions to `CoroutineScope`, so an implicit or explicit receiver must always + be passed when you call them. +* The coroutine started by `runBlocking` is the only exception because `runBlocking` is defined as a top-level function. + But because it blocks the current thread, it's intended primarily to be used in `main()` functions and tests as a bridge + function. + +A new coroutine inside `runBlocking`, `launch`, or `async` is started automatically inside the scope: + +```kotlin +import kotlinx.coroutines.* + +fun main() = runBlocking { /* this: CoroutineScope */ + launch { /* ... */ } + // the same as: + this.launch { /* ... */ } +} +``` + +When you call `launch` inside `runBlocking`, it's called as an extension to the implicit receiver of +the `CoroutineScope` type. Alternatively, you could explicitly write `this.launch`. + +The nested coroutine (started by `launch` in this example) can be considered as a child of the outer coroutine (started +by `runBlocking`). This "parent-child" relationship works through scopes; the child coroutine is started from the scope +corresponding to the parent coroutine. + +It's possible to create a new scope without starting a new coroutine, by using the `coroutineScope` function. +To start new coroutines in a structured way inside a `suspend` function without access to the outer scope, you can create +a new coroutine scope that automatically becomes a child of the outer scope that this `suspend` function is called from. +`loadContributorsConcurrent()`is a good example. + +You can also start a new coroutine from the global scope using `GlobalScope.async` or `GlobalScope.launch`. +This will create a top-level "independent" coroutine. + +The mechanism behind the structure of the coroutines is called _structured concurrency_. It provides the following +benefits over global scopes: + +* The scope is generally responsible for child coroutines, whose lifetime is attached to the lifetime of the scope. +* The scope can automatically cancel child coroutines if something goes wrong or a user changes their mind and decides + to revoke the operation. +* The scope automatically waits for the completion of all child coroutines. + Therefore, if the scope corresponds to a coroutine, the parent coroutine does not complete until all the coroutines + launched in its scope have completed. + +When using `GlobalScope.async`, there is no structure that binds several coroutines to a smaller scope. +Coroutines started from the global scope are all independent – their lifetime is limited only by the lifetime of the +whole application. It's possible to store a reference to the coroutine started from the global scope and wait for its +completion or cancel it explicitly, but that won't happen automatically as it would with structured concurrency. + +### Canceling the loading of contributors + +Create two versions of the function that loads the list of contributors. Compare how both versions behave when you try to +cancel the parent coroutine. The first version will use `coroutineScope` to start all of the child coroutines, +whereas the second will use `GlobalScope`. + +1. In `Request5Concurrent.kt`, add a 3-second delay to the `loadContributorsConcurrent()` function: + + ```kotlin + suspend fun loadContributorsConcurrent( + service: GitHubService, + req: RequestData + ): List<User> = coroutineScope { + // ... + async { + log("starting loading for ${repo.name}") + delay(3000) + // load repo contributors + } + // ... + } + ``` + + The delay affects all of the coroutines that send requests, so that there's enough time to cancel the loading + after the coroutines are started but before the requests are sent. + +2. Create the second version of the loading function: copy the implementation of `loadContributorsConcurrent()` to + `loadContributorsNotCancellable()` in `Request5NotCancellable.kt` and then remove the creation of a new `coroutineScope`. +3. The `async` calls now fail to resolve, so start them by using `GlobalScope.async`: + + ```kotlin + suspend fun loadContributorsNotCancellable( + service: GitHubService, + req: RequestData + ): List<User> { // #1 + // ... + GlobalScope.async { // #2 + log("starting loading for ${repo.name}") + // load repo contributors + } + // ... + return deferreds.awaitAll().flatten().aggregate() // #3 + } + ``` + + * The function now returns the result directly, not as the last expression inside the lambda (lines `#1` and `#3`). + * All of the "contributors" coroutines are started inside the `GlobalScope`, not as children of the coroutine scope + (line `#2`). + +4. Run the program and choose the _CONCURRENT_ option to load the contributors. +5. Wait until all of the "contributors" coroutines are started, and then click _Cancel_. The log shows no new results, + which means that all of the requests were indeed canceled: + + ```text + 2896 [AWT-EventQueue-0 @coroutine#1] INFO Contributors - kotlin: loaded 40 repos + 2901 [DefaultDispatcher-worker-2 @coroutine#4] INFO Contributors - starting loading for kotlin-koans + ... + 2909 [DefaultDispatcher-worker-5 @coroutine#36] INFO Contributors - starting loading for mpp-example + /* click on 'cancel' */ + /* no requests are sent */ + ``` + +6. Repeat step 5, but this time choose the `NOT_CANCELLABLE` option: + + ```text + 2570 [AWT-EventQueue-0 @coroutine#1] INFO Contributors - kotlin: loaded 30 repos + 2579 [DefaultDispatcher-worker-1 @coroutine#4] INFO Contributors - starting loading for kotlin-koans + ... + 2586 [DefaultDispatcher-worker-6 @coroutine#36] INFO Contributors - starting loading for mpp-example + /* click on 'cancel' */ + /* but all the requests are still sent: */ + 6402 [DefaultDispatcher-worker-5 @coroutine#4] INFO Contributors - kotlin-koans: loaded 45 contributors + ... + 9555 [DefaultDispatcher-worker-8 @coroutine#36] INFO Contributors - mpp-example: loaded 8 contributors + ``` + + In this case, no coroutines are canceled, and all the requests are still sent. + +7. Check how the cancellation is triggered in the "contributors" program. When the _Cancel_ button is clicked, + the main "loading" coroutine is explicitly canceled and the child coroutines are canceled automatically: + + ```kotlin + interface Contributors { + + fun loadContributors() { + // ... + when (getSelectedVariant()) { + CONCURRENT -> { + launch { + val users = loadContributorsConcurrent(service, req) + updateResults(users, startTime) + }.setUpCancellation() // #1 + } + } + } + + private fun Job.setUpCancellation() { + val loadingJob = this // #2 + + // cancel the loading job if the 'cancel' button was clicked: + val listener = ActionListener { + loadingJob.cancel() // #3 + updateLoadingStatus(CANCELED) + } + // add a listener to the 'cancel' button: + addCancelListener(listener) + + // update the status and remove the listener + // after the loading job is completed + } + } + ``` + +The `launch` function returns an instance of `Job`. `Job` stores a reference to the "loading coroutine", which loads +all of the data and updates the results. You can call the `setUpCancellation()` extension function on it (line `#1`), +passing an instance of `Job` as a receiver. + +Another way you could express this would be to explicitly write: + +```kotlin +val job = launch { } +job.setUpCancellation() +``` + +* For readability, you could refer to the `setUpCancellation()` function receiver inside the function with the + new `loadingJob` variable (line `#2`). +* Then you could add a listener to the _Cancel_ button so that when it's clicked, the `loadingJob` is canceled (line `#3`). + +With structured concurrency, you only need to cancel the parent coroutine and this automatically propagates cancellation +to all of the child coroutines. + +### Using the outer scope's context + +When you start new coroutines inside the given scope, it's much easier to ensure that all of them run with the same +context. It is also much easier to replace the context if needed. + +Now it's time to learn how using the dispatcher from the outer scope works. The new scope created by +the `coroutineScope` or by the coroutine builders always inherits the context from the outer scope. In this case, the +outer scope is the scope the `suspend loadContributorsConcurrent()` function was called from: + +```kotlin +launch(Dispatchers.Default) { // outer scope + val users = loadContributorsConcurrent(service, req) + // ... +} +``` + +All of the nested coroutines are automatically started with the inherited context. The dispatcher is a part of this +context. That's why all of the coroutines started by `async` are started with the context of the default dispatcher: + +```kotlin +suspend fun loadContributorsConcurrent( + service: GitHubService, req: RequestData +): List<User> = coroutineScope { + // this scope inherits the context from the outer scope + // ... + async { // nested coroutine started with the inherited context + // ... + } + // ... +} +``` + +With structured concurrency, you can specify the major context elements (like dispatcher) once, when creating the +top-level coroutine. All the nested coroutines then inherit the context and modify it only if needed. + +> When you write code with coroutines for UI applications, for example Android ones, it's a common practice to +> use `CoroutineDispatchers.Main` by default for the top coroutine and then to explicitly put a different dispatcher when +> you need to run the code on a different thread. +> +{type="tip"} + +## Showing progress + +Despite the information for some repositories being loaded rather quickly, the user only sees the resulting list after all of +the data has been loaded. Until then, the loader icon runs showing the progress, but there's no information about the current +state or what contributors are already loaded. + +You can show the intermediate results earlier and display all of the contributors after loading the data for each of the +repositories: + +{width=500} + +To implement this functionality, in the `src/tasks/Request6Progress.kt`, you'll need to pass the logic updating the UI +as a callback, so that it's called on each intermediate state: + +```kotlin +suspend fun loadContributorsProgress( + service: GitHubService, + req: RequestData, + updateResults: suspend (List<User>, completed: Boolean) -> Unit +) { + // loading the data + // calling `updateResults()` on intermediate states +} +``` + +On the call site in `Contributors.kt`, the callback is passed to update the results from the `Main` thread for +the _PROGRESS_ option: + +```kotlin +launch(Dispatchers.Default) { + loadContributorsProgress(service, req) { users, completed -> + withContext(Dispatchers.Main) { + updateResults(users, startTime, completed) + } + } +} +``` + +* The `updateResults()` parameter is declared as `suspend` in `loadContributorsProgress()`. It's necessary to call + `withContext`, which is a `suspend` function inside the corresponding lambda argument. +* `updateResults()` callback takes an additional Boolean parameter as an argument specifying whether the loading has + completed and the results are final. + +### Task 6 + +In the `Request6Progress.kt` file, implement the `loadContributorsProgress()` function that shows the intermediate +progress. Base it on the `loadContributorsSuspend()` function from `Request4Suspend.kt`. + +* Use a simple version without concurrency; you'll add it later in the next section. +* The intermediate list of contributors should be shown in an "aggregated" state, not just the list of users loaded for + each repository. +* The total number of contributions for each user should be increased when the data for each new + repository is loaded. + +#### Solution for task 6 {initial-collapse-state="collapsed"} + +To store the intermediate list of loaded contributors in the "aggregated" state, define an `allUsers` variable which +stores the list of users, and then update it after contributors for each new repository are loaded: + +```kotlin +suspend fun loadContributorsProgress( + service: GitHubService, + req: RequestData, + updateResults: suspend (List<User>, completed: Boolean) -> Unit +) { + val repos = service + .getOrgRepos(req.org) + .also { logRepos(req, it) } + .bodyList() + + var allUsers = emptyList<User>() + for ((index, repo) in repos.withIndex()) { + val users = service.getRepoContributors(req.org, repo.name) + .also { logUsers(repo, it) } + .bodyList() + + allUsers = (allUsers + users).aggregate() + updateResults(allUsers, index == repos.lastIndex) + } +} +``` + +#### Consecutive vs concurrent + +An `updateResults()` callback is called after each request is completed: + +{width=700} + +This code doesn't include concurrency. It's sequential, so you don't need synchronization. + +The best option would be to send requests concurrently and update the intermediate results after getting the response +for each repository: + +{width=700} + +To add concurrency, use _channels_. + +## Channels + +Writing code with a shared mutable state is quite difficult and error-prone (like in the solution using callbacks). +A simpler way is to share information by communication rather than by using a common mutable state. +Coroutines can communicate with each other through _channels_. + +Channels are communication primitives that allow data to be passed between coroutines. One coroutine can _send_ +some information to a channel, while another can _receive_ that information from it: + + + +A coroutine that sends (produces) information is often called a producer, and a coroutine that receives (consumes) +information is called a consumer. One or multiple coroutines can send information to the same channel, and one or multiple +coroutines can receive data from it: + + + +When many coroutines receive information from the same channel, each element is handled only once by one of the +consumers. Once an element is handled, it is immediately removed from the channel. + +You can think of a channel as similar to a collection of elements, or more precisely, a queue, in which elements are added +to one end and received from the other. However, there's an important difference: unlike collections, even in their +synchronized versions, a channel can _suspend_ `send()`and `receive()` operations. This happens when the channel is empty +or full. The channel can be full if the channel size has an upper bound. + +`Channel` is represented by three different interfaces: `SendChannel`, `ReceiveChannel`, and `Channel`, with the latter +extending the first two. You usually create a channel and give it to producers as a `SendChannel` instance so that only +they can send information to the channel. +You give a channel to consumers as a `ReceiveChannel` instance so that only they can receive from it. Both `send` +and `receive` methods are declared as `suspend`: + +```kotlin +interface SendChannel<in E> { + suspend fun send(element: E) + fun close(): Boolean +} + +interface ReceiveChannel<out E> { + suspend fun receive(): E +} + +interface Channel<E> : SendChannel<E>, ReceiveChannel<E> +``` + +The producer can close a channel to indicate that no more elements are coming. + +Several types of channels are defined in the library. They differ in how many elements they can internally store and +whether the `send()` call can be suspended or not. +For all of the channel types, the `receive()` call behaves similarly: it receives an element if the channel is not empty; +otherwise, it is suspended. + +<deflist collapsible="true"> + <def title="Unlimited channel"> + <p>An unlimited channel is the closest analog to a queue: producers can send elements to this channel and it will +keep growing indefinitely. The <code>send()</code> call will never be suspended. +If the program runs out of memory, you'll get an <code>OutOfMemoryException</code>. +The difference between an unlimited channel and a queue is that when a consumer tries to receive from an empty channel, +it becomes suspended until some new elements are sent.</p> + <img src="unlimited-channel.png" alt="Unlimited channel" width="500"/> + </def> + <def title="Buffered channel"> + <p>The size of a buffered channel is constrained by the specified number. +Producers can send elements to this channel until the size limit is reached. All of the elements are internally stored. +When the channel is full, the next `send` call on it is suspended until more free space becomes available.</p> + <img src="buffered-channel.png" alt="Buffered channel" width="500"/> + </def> + <def title="Rendezvous channel"> + <p>The "Rendezvous" channel is a channel without a buffer, the same as a buffered channel with zero size. +One of the functions (<code>send()</code> or <code>receive()</code>) is always suspended until the other is called. </p> + <p>If the <code>send()</code> function is called and there's no suspended <code>receive</code> call ready to process the element, then <code>send()</code> +is suspended. Similarly, if the <code>receive</code> function is called and the channel is empty or, in other words, there's no +suspended <code>send()</code> call ready to send the element, the <code>receive()</code> call is suspended. </p> + <p>The "rendezvous" name ("a meeting at an agreed time and place") refers to the fact that <code>send()</code> and <code>receive()</code> +should "meet on time".</p> + <img src="rendezvous-channel.png" alt="Rendezvous channel" width="500"/> + </def> + <def title="Conflated channel"> + <p>A new element sent to the conflated channel will overwrite the previously sent element, so the receiver will always +get only the latest element. The <code>send()</code> call is never suspended.</p> + <img src="conflated-channel.gif" alt="Conflated channel" width="500"/> + </def> +</deflist> + +When you create a channel, specify its type or the buffer size (if you need a buffered one): + +```kotlin +val rendezvousChannel = Channel<String>() +val bufferedChannel = Channel<String>(10) +val conflatedChannel = Channel<String>(CONFLATED) +val unlimitedChannel = Channel<String>(UNLIMITED) +``` + +By default, a "Rendezvous" channel is created. + +In the following task, you'll create a "Rendezvous" channel, two producer coroutines, and a consumer coroutine: + +```kotlin +import kotlinx.coroutines.channels.Channel +import kotlinx.coroutines.* + +fun main() = runBlocking<Unit> { + val channel = Channel<String>() + launch { + channel.send("A1") + channel.send("A2") + log("A done") + } + launch { + channel.send("B1") + log("B done") + } + launch { + repeat(3) { + val x = channel.receive() + log(x) + } + } +} + +fun log(message: Any?) { + println("[${Thread.currentThread().name}] $message") +} +``` + +> Watch [this video](https://www.youtube.com/watch?v=HpWQUoVURWQ) for a better understanding of channels. +> +{type="tip"} + +### Task 7 + +In `src/tasks/Request7Channels.kt`, implement the function `loadContributorsChannels()` that requests all of the GitHub +contributors concurrently and shows intermediate progress at the same time. + +Use the previous functions, `loadContributorsConcurrent()` from `Request5Concurrent.kt` +and `loadContributorsProgress()` from `Request6Progress.kt`. + +#### Tip for task 7 {initial-collapse-state="collapsed"} + +Different coroutines that concurrently receive contributor lists for different repositories can send all of the received +results to the same channel: + +```kotlin +val channel = Channel<List<User>>() +for (repo in repos) { + launch { + val users = TODO() + // ... + channel.send(users) + } +} +``` + +Then the elements from this channel can be received one by one and processed: + +```kotlin +repeat(repos.size) { + val users = channel.receive() + // ... +} +``` + +Since the `receive()` calls are sequential, no additional synchronization is needed. + +#### Solution for task 7 {initial-collapse-state="collapsed"} + +As with the `loadContributorsProgress()` function, you can create an `allUsers` variable to store the intermediate +states of the "all contributors" list. +Each new list received from the channel is added to the list of all users. You aggregate the result and update the state +using the `updateResults` callback: + +```kotlin +suspend fun loadContributorsChannels( + service: GitHubService, + req: RequestData, + updateResults: suspend (List<User>, completed: Boolean) -> Unit +) = coroutineScope { + + val repos = service + .getOrgRepos(req.org) + .also { logRepos(req, it) } + .bodyList() + + val channel = Channel<List<User>>() + for (repo in repos) { + launch { + val users = service.getRepoContributors(req.org, repo.name) + .also { logUsers(repo, it) } + .bodyList() + channel.send(users) + } + } + var allUsers = emptyList<User>() + repeat(repos.size) { + val users = channel.receive() + allUsers = (allUsers + users).aggregate() + updateResults(allUsers, it == repos.lastIndex) + } +} +``` + +* Results for different repositories are added to the channel as soon as they are ready. At first, when all of the requests + are sent, and no data is received, the `receive()` call is suspended. In this case, the whole "load contributors" coroutine + is suspended. +* Then, when the list of users is sent to the channel, the "load contributors" coroutine resumes, the `receive()` call + returns this list, and the results are immediately updated. + +You can now run the program and choose the _CHANNELS_ option to load the contributors and see the result. + +Although neither coroutines nor channels completely remove the complexity that comes with concurrency, +they make life easier when you need to understand what's going on. + +## Testing coroutines + +Let's now test all solutions to check that the solution with concurrent coroutines is faster than the solution with +the `suspend` functions, and check that the solution with channels is faster than the simple "progress" one. + +In the following task, you'll compare the total running time of the solutions. You'll mock a GitHub service and make +this service return results after the given timeouts: + +```text +repos request - returns an answer within 1000 ms delay +repo-1 - 1000 ms delay +repo-2 - 1200 ms delay +repo-3 - 800 ms delay +``` + +The sequential solution with the `suspend` functions should take around 4000 ms (4000 = 1000 + (1000 + 1200 + 800)). +The concurrent solution should take around 2200 ms (2200 = 1000 + max(1000, 1200, 800)). + +For the solutions that show progress, you can also check the intermediate results with timestamps. + +The corresponding test data is defined in `test/contributors/testData.kt`, and the files `Request4SuspendKtTest`, +`Request7ChannelsKtTest`, and so on contain the straightforward tests that use mock service calls. + +However, there are two problems here: + +* These tests take too long to run. Each test takes around 2 to 4 seconds, and you need to wait for the results each + time. It's not very efficient. +* You can't rely on the exact time the solution runs because it still takes additional time to prepare and run the code. + You could add a constant, but then the time would differ from machine to machine. The mock service delays + should be higher than this constant so you can see a difference. If the constant is 0.5 sec, making the delays + 0.1 sec won't be enough. + +A better way would be to use special frameworks to test the timing while running the same code several times (which increases +the total time even more), but that is complicated to learn and set up. + +To solve these problems and make sure that solutions with provided test delays behave as expected, one faster than the other, +use _virtual_ time with a special test dispatcher. This dispatcher keeps track of the virtual time passed from +the start and runs everything immediately in real time. When you run coroutines on this dispatcher, +the `delay` will return immediately and advance the virtual time. + +Tests that use this mechanism run fast, but you can still check what happens at different moments in virtual time. The +total running time drastically decreases: + +{width=700} + +To use virtual time, replace the `runBlocking` invocation with a `runTest`. `runTest` takes an +extension lambda to `TestScope` as an argument. +When you call `delay` in a `suspend` function inside this special scope, `delay` will increase the virtual time instead +of delaying in real time: + +```kotlin +@Test +fun testDelayInSuspend() = runTest { + val realStartTime = System.currentTimeMillis() + val virtualStartTime = currentTime + + foo() + println("${System.currentTimeMillis() - realStartTime} ms") // ~ 6 ms + println("${currentTime - virtualStartTime} ms") // 1000 ms +} + +suspend fun foo() { + delay(1000) // auto-advances without delay + println("foo") // executes eagerly when foo() is called +} +``` + +You can check the current virtual time using the `currentTime` property of `TestScope`. + +The actual running time in this example is several milliseconds, whereas virtual time equals the delay argument, which +is 1000 milliseconds. + +To get the full effect of "virtual" `delay` in child coroutines, +start all of the child coroutines with `TestDispatcher`. Otherwise, it won't work. This dispatcher is +automatically inherited from the other `TestScope`, unless you provide a different dispatcher: + +```kotlin +@Test +fun testDelayInLaunch() = runTest { + val realStartTime = System.currentTimeMillis() + val virtualStartTime = currentTime + + bar() + + println("${System.currentTimeMillis() - realStartTime} ms") // ~ 11 ms + println("${currentTime - virtualStartTime} ms") // 1000 ms +} + +suspend fun bar() = coroutineScope { + launch { + delay(1000) // auto-advances without delay + println("bar") // executes eagerly when bar() is called + } +} +``` + +If `launch` is called with the context of `Dispatchers.Default` in the example above, the test will fail. You'll get an +exception saying that the job has not been completed yet. + +You can test the `loadContributorsConcurrent()` function this way only if it starts the child coroutines with the +inherited context, without modifying it using the `Dispatchers.Default` dispatcher. + +You can specify the context elements like the dispatcher when _calling_ a function rather than when _defining_ it, +which allows for more flexibility and easier testing. + +> The testing API that supports virtual time is [Experimental](components-stability.md) and may change in the future. +> +{type="warning"} + +By default, the compiler shows warnings if you use the experimental testing API. To suppress these warnings, annotate +the test function or the whole class containing the tests with `@OptIn(ExperimentalCoroutinesApi::class)`. +Add the compiler argument instructing the compiler that you're using the experimental API: + +```kotlin +compileTestKotlin { + kotlinOptions { + freeCompilerArgs += "-Xuse-experimental=kotlin.Experimental" + } +} +``` + +In the project corresponding to this tutorial, the compiler argument has already been added to the Gradle script. + +### Task 8 + +Refactor the following tests in `tests/tasks/` to use virtual time instead of real time: + +* Request4SuspendKtTest.kt +* Request5ConcurrentKtTest.kt +* Request6ProgressKtTest.kt +* Request7ChannelsKtTest.kt + +Compare the total running times before and after applying your refactoring. + +#### Tip for task 8 {initial-collapse-state="collapsed"} + +1. Replace the `runBlocking` invocation with `runTest`, and replace `System.currentTimeMillis()` with `currentTime`: + + ```kotlin + @Test + fun test() = runTest { + val startTime = currentTime + // action + val totalTime = currentTime - startTime + // testing result + } + ``` + +2. Uncomment the assertions that check the exact virtual time. +3. Don't forget to add `@UseExperimental(ExperimentalCoroutinesApi::class)`. + +#### Solution for task 8 {initial-collapse-state="collapsed"} + +Here are the solutions for the concurrent and channels cases: + +```kotlin +fun testConcurrent() = runTest { + val startTime = currentTime + val result = loadContributorsConcurrent(MockGithubService, testRequestData) + Assert.assertEquals("Wrong result for 'loadContributorsConcurrent'", expectedConcurrentResults.users, result) + val totalTime = currentTime - startTime + + Assert.assertEquals( + "The calls run concurrently, so the total virtual time should be 2200 ms: " + + "1000 for repos request plus max(1000, 1200, 800) = 1200 for concurrent contributors requests)", + expectedConcurrentResults.timeFromStart, totalTime + ) +} +``` + +First, check that the results are available exactly at the expected virtual time, and then check the results +themselves: + +```kotlin +fun testChannels() = runTest { + val startTime = currentTime + var index = 0 + loadContributorsChannels(MockGithubService, testRequestData) { users, _ -> + val expected = concurrentProgressResults[index++] + val time = currentTime - startTime + Assert.assertEquals( + "Expected intermediate results after ${expected.timeFromStart} ms:", + expected.timeFromStart, time + ) + Assert.assertEquals("Wrong intermediate results after $time:", expected.users, users) + } +} +``` + +The first intermediate result for the last version with channels becomes available sooner than the progress version, and you +can see the difference in tests that use virtual time. + +> The tests for the remaining "suspend" and "progress" tasks are very similar – you can find them in the project's +> `solutions` branch. +> +{type="tip"} + +## What's next + +* Check out the [Asynchronous Programming with Kotlin](https://kotlinconf.com/workshops/) workshop at KotlinConf. +* Find out more about using [virtual time and the experimental testing package](https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-test/). diff --git a/docs/topics/coroutines-basics.md b/docs/topics/coroutines-basics.md index ad33dcfb..d7d67003 100644 --- a/docs/topics/coroutines-basics.md +++ b/docs/topics/coroutines-basics.md @@ -57,7 +57,7 @@ the code with coroutines inside of `runBlocking { ... }` curly braces. This is h `this: CoroutineScope` hint right after the `runBlocking` opening curly brace. If you remove or forget `runBlocking` in this code, you'll get an error on the [launch] call, since `launch` -is declared only in the [CoroutineScope]: +is declared only on the [CoroutineScope]: ```Plain Text Unresolved reference: launch @@ -71,7 +71,7 @@ as threads are expensive resources and blocking them is inefficient and is often ### Structured concurrency Coroutines follow a principle of -**structured concurrency** which means that new coroutines can be only launched in a specific [CoroutineScope] +**structured concurrency** which means that new coroutines can only be launched in a specific [CoroutineScope] which delimits the lifetime of the coroutine. The above example shows that [runBlocking] establishes the corresponding scope and that is why the previous example waits until `World!` is printed after a second's delay and only then exits. @@ -250,14 +250,14 @@ Done Coroutines are less resource-intensive than JVM threads. Code that exhausts the JVM's available memory when using threads can be expressed using coroutines without hitting resource limits. For example, the following code launches -100000 distinct coroutines that each wait 5 seconds and then print a period +50,000 distinct coroutines that each waits 5 seconds and then prints a period ('.') while consuming very little memory: ```kotlin import kotlinx.coroutines.* fun main() = runBlocking { - repeat(100_000) { // launch a lot of coroutines + repeat(50_000) { // launch a lot of coroutines launch { delay(5000L) print(".") @@ -265,18 +265,19 @@ fun main() = runBlocking { } } ``` -<!-- While coroutines do have a smaller memory footprint than threads, this -example will exhaust the playground's heap memory; don't make it runnable. --> +{kotlin-runnable="true" kotlin-min-compiler-version="1.3"} > You can get the full code [here](../../kotlinx-coroutines-core/jvm/test/guide/example-basic-06.kt). > {type="note"} -<!--- TEST lines.size == 1 && lines[0] == ".".repeat(100_000) --> +<!--- TEST lines.size == 1 && lines[0] == ".".repeat(50_000) --> If you write the same program using threads (remove `runBlocking`, replace `launch` with `thread`, and replace `delay` with `Thread.sleep`), it will -likely consume too much memory and throw an out-of-memory error. +consume a lot of memory. Depending on your operating system, JDK version, +and its settings, it will either throw an out-of-memory error or start threads slowly +so that there are never too many concurrently running threads. <!--- MODULE kotlinx-coroutines-core --> <!--- INDEX kotlinx.coroutines --> diff --git a/docs/topics/coroutines-guide.md b/docs/topics/coroutines-guide.md index 73c4c1ac..fd95c38d 100644 --- a/docs/topics/coroutines-guide.md +++ b/docs/topics/coroutines-guide.md @@ -1,15 +1,15 @@ [//]: # (title: Coroutines guide) -Kotlin, as a language, provides only minimal low-level APIs in its standard library to enable various other +Kotlin provides only minimal low-level APIs in its standard library to enable other libraries to utilize coroutines. Unlike many other languages with similar capabilities, `async` and `await` are not keywords in Kotlin and are not even part of its standard library. Moreover, Kotlin's concept of _suspending function_ provides a safer and less error-prone abstraction for asynchronous operations than futures and promises. `kotlinx.coroutines` is a rich library for coroutines developed by JetBrains. It contains a number of high-level -coroutine-enabled primitives that this guide covers, including `launch`, `async` and others. +coroutine-enabled primitives that this guide covers, including `launch`, `async`, and others. -This is a guide on core features of `kotlinx.coroutines` with a series of examples, divided up into different topics. +This is a guide about the core features of `kotlinx.coroutines` with a series of examples, divided up into different topics. In order to use coroutines as well as follow the examples in this guide, you need to add a dependency on the `kotlinx-coroutines-core` module as explained [in the project README](https://github.com/Kotlin/kotlinx.coroutines/blob/master/README.md#using-in-your-projects). diff --git a/docs/topics/debug-coroutines-with-idea.md b/docs/topics/debug-coroutines-with-idea.md index 2541c924..b31aa79f 100644 --- a/docs/topics/debug-coroutines-with-idea.md +++ b/docs/topics/debug-coroutines-with-idea.md @@ -4,19 +4,39 @@ This tutorial demonstrates how to create Kotlin coroutines and debug them using The tutorial assumes you have prior knowledge of the [coroutines](coroutines-guide.md) concept. -> Debugging works for `kotlinx-coroutines-core` version 1.3.8 or later. -> -{type="note"} - ## Create coroutines -1. Open a Kotlin project in IntelliJ IDEA. If you don't have a project, [create one](jvm-get-started.md#create-an-application). +1. Open a Kotlin project in IntelliJ IDEA. If you don't have a project, [create one](jvm-get-started.md#create-a-project). +2. To use the `kotlinx.coroutines` library in a Gradle project, add the following dependency to `build.gradle(.kts)`: + + <tabs group="build-script"> + <tab title="Kotlin" group-key="kotlin"> -2. Open the `main.kt` file in `src/main/kotlin`. + ```kotlin + dependencies { + implementation("org.jetbrains.kotlinx:kotlinx-coroutines-core:%coroutinesVersion%") + } + ``` - The `src` directory contains Kotlin source files and resources. The `main.kt` file contains sample code that will print `Hello World!`. + </tab> + <tab title="Groovy" group-key="groovy"> + + ```groovy + dependencies { + implementation 'org.jetbrains.kotlinx:kotlinx-coroutines-core:%coroutinesVersion%' + } + ``` + + </tab> + </tabs> -3. Change code in the `main()` function: + For other build systems, see instructions in the [`kotlinx.coroutines` README](https://github.com/Kotlin/kotlinx.coroutines#using-in-your-projects). + +3. Open the `Main.kt` file in `src/main/kotlin`. + + The `src` directory contains Kotlin source files and resources. The `Main.kt` file contains sample code that will print `Hello World!`. + +4. Change code in the `main()` function: * Use the [`runBlocking()`](https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/run-blocking.html) block to wrap a coroutine. * Use the [`async()`](https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/async.html) function to create coroutines that compute deferred values `a` and `b`. @@ -39,7 +59,7 @@ The tutorial assumes you have prior knowledge of the [coroutines](coroutines-gui } ``` -4. Build the code by clicking **Build Project**. +5. Build the code by clicking **Build Project**.  @@ -80,3 +100,17 @@ The tutorial assumes you have prior knowledge of the [coroutines](coroutines-gui * The third coroutine is calculating the value of `b` – it has the **RUNNING** status. Using IntelliJ IDEA debugger, you can dig deeper into each coroutine to debug your code. + +### Optimized-out variables + +If you use `suspend` functions, in the debugger, you might see the "was optimized out" text next to a variable's name: + + + +This text means that the variable's lifetime was decreased, and the variable doesn't exist anymore. +It is difficult to debug code with optimized variables because you don't see their values. +You can disable this behavior with the `-Xdebug` compiler option. + +> __Never use this flag in production__: `-Xdebug` can [cause memory leaks](https://youtrack.jetbrains.com/issue/KT-48678/Coroutine-debugger-disable-was-optimised-out-compiler-feature#focus=Comments-27-6015585.0-0). +> +{type="warning"}
\ No newline at end of file diff --git a/docs/topics/debug-flow-with-idea.md b/docs/topics/debug-flow-with-idea.md index b769e795..0aa78b17 100644 --- a/docs/topics/debug-flow-with-idea.md +++ b/docs/topics/debug-flow-with-idea.md @@ -4,21 +4,41 @@ This tutorial demonstrates how to create Kotlin Flow and debug it using IntelliJ The tutorial assumes you have prior knowledge of the [coroutines](coroutines-guide.md) and [Kotlin Flow](flow.md#flows) concepts. -> Debugging works for `kotlinx-coroutines-core` version 1.3.8 or later. -> -{type="note"} - ## Create a Kotlin flow Create a Kotlin [flow](https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.flow/flow.html) with a slow emitter and a slow collector: -1. Open a Kotlin project in IntelliJ IDEA. If you don't have a project, [create one](jvm-get-started.md#create-an-application). - -2. Open the `main.kt` file in `src/main/kotlin`. - - The `src` directory contains Kotlin source files and resources. The `main.kt` file contains sample code that will print `Hello World!`. - -3. Create the `simple()` function that returns a flow of three numbers: +1. Open a Kotlin project in IntelliJ IDEA. If you don't have a project, [create one](jvm-get-started.md#create-a-project). +2. To use the `kotlinx.coroutines` library in a Gradle project, add the following dependency to `build.gradle(.kts)`: + + <tabs group="build-script"> + <tab title="Kotlin" group-key="kotlin"> + + ```kotlin + dependencies { + implementation("org.jetbrains.kotlinx:kotlinx-coroutines-core:%coroutinesVersion%") + } + ``` + + </tab> + <tab title="Groovy" group-key="groovy"> + + ```groovy + dependencies { + implementation 'org.jetbrains.kotlinx:kotlinx-coroutines-core:%coroutinesVersion%' + } + ``` + + </tab> + </tabs> + + For other build systems, see instructions in the [`kotlinx.coroutines` README](https://github.com/Kotlin/kotlinx.coroutines#using-in-your-projects). + +3. Open the `Main.kt` file in `src/main/kotlin`. + + The `src` directory contains Kotlin source files and resources. The `Main.kt` file contains sample code that will print `Hello World!`. + +4. Create the `simple()` function that returns a flow of three numbers: * Use the [`delay()`](https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/delay.html) function to imitate CPU-consuming blocking code. It suspends the coroutine for 100 ms without blocking the thread. * Produce the values in the `for` loop using the [`emit()`](https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.flow/-flow-collector/emit.html) function. @@ -36,12 +56,12 @@ Create a Kotlin [flow](https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-cor } ``` -4. Change the code in the `main()` function: +5. Change the code in the `main()` function: * Use the [`runBlocking()`](https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/run-blocking.html) block to wrap a coroutine. * Collect the emitted values using the [`collect()`](https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.flow/collect.html) function. * Use the [`delay()`](https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/delay.html) function to imitate CPU-consuming code. It suspends the coroutine for 300 ms without blocking the thread. - * Print the collected value from the flow using the [`println()`](https://kotlinlang.org/api/latest/jvm/stdlib/stdlib/kotlin.io/println.html) function. + * Print the collected value from the flow using the [`println()`](https://kotlinlang.org/api/latest/jvm/stdlib/kotlin.io/println.html) function. ```kotlin fun main() = runBlocking { @@ -53,7 +73,7 @@ Create a Kotlin [flow](https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-cor } ``` -5. Build the code by clicking **Build Project**. +6. Build the code by clicking **Build Project**.  @@ -82,9 +102,23 @@ Create a Kotlin [flow](https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-cor  +### Optimized-out variables + +If you use `suspend` functions, in the debugger, you might see the "was optimized out" text next to a variable's name: + + + +This text means that the variable's lifetime was decreased, and the variable doesn't exist anymore. +It is difficult to debug code with optimized variables because you don't see their values. +You can disable this behavior with the `-Xdebug` compiler option. + +> __Never use this flag in production__: `-Xdebug` can [cause memory leaks](https://youtrack.jetbrains.com/issue/KT-48678/Coroutine-debugger-disable-was-optimised-out-compiler-feature#focus=Comments-27-6015585.0-0). +> +{type="warning"} + ## Add a concurrently running coroutine -1. Open the `main.kt` file in `src/main/kotlin`. +1. Open the `Main.kt` file in `src/main/kotlin`. 2. Enhance the code to run the emitter and collector concurrently: diff --git a/docs/topics/flow.md b/docs/topics/flow.md index 538a9d67..b678b70a 100644 --- a/docs/topics/flow.md +++ b/docs/topics/flow.md @@ -102,7 +102,7 @@ This code prints the numbers after waiting for a second. ### Flows Using the `List<Int>` result type, means we can only return all the values at once. To represent -the stream of values that are being asynchronously computed, we can use a [`Flow<Int>`][Flow] type just like we would use the `Sequence<Int>` type for synchronously computed values: +the stream of values that are being computed asynchronously, we can use a [`Flow<Int>`][Flow] type just like we would use a `Sequence<Int>` type for synchronously computed values: ```kotlin import kotlinx.coroutines.* @@ -151,11 +151,11 @@ I'm not blocked 3 Notice the following differences in the code with the [Flow] from the earlier examples: -* A builder function for [Flow] type is called [flow][_flow]. -* Code inside the `flow { ... }` builder block can suspend. -* The `simple` function is no longer marked with `suspend` modifier. -* Values are _emitted_ from the flow using [emit][FlowCollector.emit] function. -* Values are _collected_ from the flow using [collect][collect] function. +* A builder function of [Flow] type is called [flow][_flow]. +* Code inside a `flow { ... }` builder block can suspend. +* The `simple` function is no longer marked with a `suspend` modifier. +* Values are _emitted_ from the flow using an [emit][FlowCollector.emit] function. +* Values are _collected_ from the flow using a [collect][collect] function. > We can replace [delay] with `Thread.sleep` in the body of `simple`'s `flow { ... }` and see that the main > thread is blocked in this case. @@ -215,12 +215,12 @@ Flow started <!--- TEST --> This is a key reason the `simple` function (which returns a flow) is not marked with `suspend` modifier. -By itself, `simple()` call returns quickly and does not wait for anything. The flow starts every time it is collected, -that is why we see "Flow started" when we call `collect` again. +The `simple()` call itself returns quickly and does not wait for anything. The flow starts afresh every time it is +collected and that is why we see "Flow started" every time we call `collect` again. ## Flow cancellation basics -Flow adheres to the general cooperative cancellation of coroutines. As usual, flow collection can be +Flows adhere to the general cooperative cancellation of coroutines. As usual, flow collection can be cancelled when the flow is suspended in a cancellable suspending function (like [delay]). The following example shows how the flow gets cancelled on a timeout when running in a [withTimeoutOrNull] block and stops executing its code: @@ -268,13 +268,13 @@ See [Flow cancellation checks](#flow-cancellation-checks) section for more detai ## Flow builders -The `flow { ... }` builder from the previous examples is the most basic one. There are other builders for -easier declaration of flows: +The `flow { ... }` builder from the previous examples is the most basic one. There are other builders +that allow flows to be declared: -* [flowOf] builder that defines a flow emitting a fixed set of values. -* Various collections and sequences can be converted to flows using `.asFlow()` extension functions. +* The [flowOf] builder defines a flow that emits a fixed set of values. +* Various collections and sequences can be converted to flows using the `.asFlow()` extension function. -So, the example that prints the numbers from 1 to 3 from a flow can be written as: +For example, the snippet that prints the numbers 1 to 3 from a flow can be rewritten as follows: ```kotlin import kotlinx.coroutines.* @@ -301,17 +301,18 @@ fun main() = runBlocking<Unit> { ## Intermediate flow operators -Flows can be transformed with operators, just as you would with collections and sequences. +Flows can be transformed using operators, in the same way as you would transform collections and +sequences. Intermediate operators are applied to an upstream flow and return a downstream flow. These operators are cold, just like flows are. A call to such an operator is not a suspending function itself. It works quickly, returning the definition of a new transformed flow. The basic operators have familiar names like [map] and [filter]. -The important difference to sequences is that blocks of +An important difference of these operators from sequences is that blocks of code inside these operators can call suspending functions. For example, a flow of incoming requests can be -mapped to the results with the [map] operator, even when performing a request is a long-running +mapped to its results with a [map] operator, even when performing a request is a long-running operation that is implemented by a suspending function: ```kotlin @@ -337,7 +338,7 @@ fun main() = runBlocking<Unit> { > {type="note"} -It produces the following three lines, each line appearing after each second: +It produces the following three lines, each appearing one second after the previous: ```text response 1 @@ -593,7 +594,7 @@ Since `simple().collect` is called from the main thread, the body of `simple`'s This is the perfect default for fast-running or asynchronous code that does not care about the execution context and does not block the caller. -### Wrong emission withContext +### A common pitfall when using withContext However, the long-running CPU-consuming code might need to be executed in the context of [Dispatchers.Default] and UI-updating code might need to be executed in the context of [Dispatchers.Main]. Usually, [withContext] is used @@ -1012,8 +1013,8 @@ We get quite a different output, where a line is printed at each emission from e ## Flattening flows -Flows represent asynchronously received sequences of values, so it is quite easy to get in a situation where -each value triggers a request for another sequence of values. For example, we can have the following +Flows represent asynchronously received sequences of values, and so it is quite easy to get into a situation +where each value triggers a request for another sequence of values. For example, we can have the following function that returns a flow of two strings 500 ms apart: ```kotlin @@ -1026,7 +1027,7 @@ fun requestFlow(i: Int): Flow<String> = flow { <!--- CLEAR --> -Now if we have a flow of three integers and call `requestFlow` for each of them like this: +Now if we have a flow of three integers and call `requestFlow` on each of them like this: ```kotlin (1..3).asFlow().map { requestFlow(it) } @@ -1034,15 +1035,15 @@ Now if we have a flow of three integers and call `requestFlow` for each of them <!--- CLEAR --> -Then we end up with a flow of flows (`Flow<Flow<String>>`) that needs to be _flattened_ into a single flow for +Then we will end up with a flow of flows (`Flow<Flow<String>>`) that needs to be _flattened_ into a single flow for further processing. Collections and sequences have [flatten][Sequence.flatten] and [flatMap][Sequence.flatMap] operators for this. However, due to the asynchronous nature of flows they call for different _modes_ of flattening, -as such, there is a family of flattening operators on flows. +and hence, a family of flattening operators on flows exists. ### flatMapConcat -Concatenating mode is implemented by [flatMapConcat] and [flattenConcat] operators. They are the most direct -analogues of the corresponding sequence operators. They wait for the inner flow to complete before +Concatenation of flows of flows is provided by the [flatMapConcat] and [flattenConcat] operators. They are the +most direct analogues of the corresponding sequence operators. They wait for the inner flow to complete before starting to collect the next one as the following example shows: ```kotlin @@ -1058,7 +1059,7 @@ fun requestFlow(i: Int): Flow<String> = flow { fun main() = runBlocking<Unit> { //sampleStart val startTime = System.currentTimeMillis() // remember the start time - (1..3).asFlow().onEach { delay(100) } // a number every 100 ms + (1..3).asFlow().onEach { delay(100) } // emit a number every 100 ms .flatMapConcat { requestFlow(it) } .collect { value -> // collect and print println("$value at ${System.currentTimeMillis() - startTime} ms from start") @@ -1087,7 +1088,7 @@ The sequential nature of [flatMapConcat] is clearly seen in the output: ### flatMapMerge -Another flattening mode is to concurrently collect all the incoming flows and merge their values into +Another flattening operation is to concurrently collect all the incoming flows and merge their values into a single flow so that values are emitted as soon as possible. It is implemented by [flatMapMerge] and [flattenMerge] operators. They both accept an optional `concurrency` parameter that limits the number of concurrent flows that are collected at the same time @@ -1141,9 +1142,9 @@ The concurrent nature of [flatMapMerge] is obvious: ### flatMapLatest -In a similar way to the [collectLatest] operator, that was shown in -["Processing the latest value"](#processing-the-latest-value) section, there is the corresponding "Latest" -flattening mode where a collection of the previous flow is cancelled as soon as new flow is emitted. +In a similar way to the [collectLatest] operator, that was described in the section +["Processing the latest value"](#processing-the-latest-value), there is the corresponding "Latest" +flattening mode where the collection of the previous flow is cancelled as soon as new flow is emitted. It is implemented by the [flatMapLatest] operator. ```kotlin @@ -1184,9 +1185,11 @@ The output here in this example is a good demonstration of how [flatMapLatest] w <!--- TEST ARBITRARY_TIME --> -> Note that [flatMapLatest] cancels all the code in its block (`{ requestFlow(it) }` in this example) on a new value. +> Note that [flatMapLatest] cancels all the code in its block (`{ requestFlow(it) }` in this example) when a new value +> is received. > It makes no difference in this particular example, because the call to `requestFlow` itself is fast, not-suspending, -> and cannot be cancelled. However, it would show up if we were to use suspending functions like `delay` in there. +> and cannot be cancelled. However, a differnce in output would be visible if we were to use suspending functions +> like `delay` in `requestFlow`. > {type="note"} diff --git a/docs/topics/select-expression.md b/docs/topics/select-expression.md index f3055737..0e95ab63 100644 --- a/docs/topics/select-expression.md +++ b/docs/topics/select-expression.md @@ -13,23 +13,23 @@ the first one that becomes available. ## Selecting from channels -Let us have two producers of strings: `fizz` and `buzz`. The `fizz` produces "Fizz" string every 300 ms: +Let us have two producers of strings: `fizz` and `buzz`. The `fizz` produces "Fizz" string every 500 ms: ```kotlin fun CoroutineScope.fizz() = produce<String> { - while (true) { // sends "Fizz" every 300 ms - delay(300) + while (true) { // sends "Fizz" every 500 ms + delay(500) send("Fizz") } } ``` -And the `buzz` produces "Buzz!" string every 500 ms: +And the `buzz` produces "Buzz!" string every 1000 ms: ```kotlin fun CoroutineScope.buzz() = produce<String> { - while (true) { // sends "Buzz!" every 500 ms - delay(500) + while (true) { // sends "Buzz!" every 1000 ms + delay(1000) send("Buzz!") } } @@ -62,15 +62,15 @@ import kotlinx.coroutines.channels.* import kotlinx.coroutines.selects.* fun CoroutineScope.fizz() = produce<String> { - while (true) { // sends "Fizz" every 300 ms - delay(300) + while (true) { // sends "Fizz" every 500 ms + delay(500) send("Fizz") } } fun CoroutineScope.buzz() = produce<String> { - while (true) { // sends "Buzz!" every 500 ms - delay(500) + while (true) { // sends "Buzz!" every 1000 ms + delay(1000) send("Buzz!") } } @@ -112,7 +112,7 @@ fizz -> 'Fizz' fizz -> 'Fizz' buzz -> 'Buzz!' fizz -> 'Fizz' -buzz -> 'Buzz!' +fizz -> 'Fizz' ``` <!--- TEST --> diff --git a/docs/topics/shared-mutable-state-and-concurrency.md b/docs/topics/shared-mutable-state-and-concurrency.md index 99cc42bc..fad13d64 100644 --- a/docs/topics/shared-mutable-state-and-concurrency.md +++ b/docs/topics/shared-mutable-state-and-concurrency.md @@ -130,7 +130,7 @@ Completed 100000 actions in Counter = --> -This code works slower, but we still don't get "Counter = 100000" at the end, because volatile variables guarantee +This code works slower, but we still don't always get "Counter = 100000" at the end, because volatile variables guarantee linearizable (this is a technical term for "atomic") reads and writes to the corresponding variable, but do not provide atomicity of larger actions (increment in our case). @@ -195,8 +195,7 @@ state or to complex operations that do not have ready-to-use thread-safe impleme _Thread confinement_ is an approach to the problem of shared mutable state where all access to the particular shared state is confined to a single thread. It is typically used in UI applications, where all UI state is confined to -the single event-dispatch/application thread. It is easy to apply with coroutines by using a -single-threaded context. +the single event-dispatch/application thread. It is easy to apply with coroutines by using a single-threaded context. <!--- CLEAR --> @@ -370,133 +369,11 @@ The locking in this example is fine-grained, so it pays the price. However, it i where you absolutely must modify some shared state periodically, but there is no natural thread that this state is confined to. -## Actors - -An [actor](https://en.wikipedia.org/wiki/Actor_model) is an entity made up of a combination of a coroutine, -the state that is confined and encapsulated into this coroutine, -and a channel to communicate with other coroutines. A simple actor can be written as a function, -but an actor with a complex state is better suited for a class. - -There is an [actor] coroutine builder that conveniently combines actor's mailbox channel into its -scope to receive messages from and combines the send channel into the resulting job object, so that a -single reference to the actor can be carried around as its handle. - -The first step of using an actor is to define a class of messages that an actor is going to process. -Kotlin's [sealed classes](https://kotlinlang.org/docs/reference/sealed-classes.html) are well suited for that purpose. -We define `CounterMsg` sealed class with `IncCounter` message to increment a counter and `GetCounter` message -to get its value. The latter needs to send a response. A [CompletableDeferred] communication -primitive, that represents a single value that will be known (communicated) in the future, -is used here for that purpose. - -```kotlin -// Message types for counterActor -sealed class CounterMsg -object IncCounter : CounterMsg() // one-way message to increment counter -class GetCounter(val response: CompletableDeferred<Int>) : CounterMsg() // a request with reply -``` - -Then we define a function that launches an actor using an [actor] coroutine builder: - -```kotlin -// This function launches a new counter actor -fun CoroutineScope.counterActor() = actor<CounterMsg> { - var counter = 0 // actor state - for (msg in channel) { // iterate over incoming messages - when (msg) { - is IncCounter -> counter++ - is GetCounter -> msg.response.complete(counter) - } - } -} -``` - -The main code is straightforward: - -<!--- CLEAR --> - -```kotlin -import kotlinx.coroutines.* -import kotlinx.coroutines.channels.* -import kotlin.system.* - -suspend fun massiveRun(action: suspend () -> Unit) { - val n = 100 // number of coroutines to launch - val k = 1000 // times an action is repeated by each coroutine - val time = measureTimeMillis { - coroutineScope { // scope for coroutines - repeat(n) { - launch { - repeat(k) { action() } - } - } - } - } - println("Completed ${n * k} actions in $time ms") -} - -// Message types for counterActor -sealed class CounterMsg -object IncCounter : CounterMsg() // one-way message to increment counter -class GetCounter(val response: CompletableDeferred<Int>) : CounterMsg() // a request with reply - -// This function launches a new counter actor -fun CoroutineScope.counterActor() = actor<CounterMsg> { - var counter = 0 // actor state - for (msg in channel) { // iterate over incoming messages - when (msg) { - is IncCounter -> counter++ - is GetCounter -> msg.response.complete(counter) - } - } -} - -//sampleStart -fun main() = runBlocking<Unit> { - val counter = counterActor() // create the actor - withContext(Dispatchers.Default) { - massiveRun { - counter.send(IncCounter) - } - } - // send a message to get a counter value from an actor - val response = CompletableDeferred<Int>() - counter.send(GetCounter(response)) - println("Counter = ${response.await()}") - counter.close() // shutdown the actor -} -//sampleEnd -``` -{kotlin-runnable="true" kotlin-min-compiler-version="1.3"} - -> You can get the full code [here](../../kotlinx-coroutines-core/jvm/test/guide/example-sync-07.kt). -> -{type="note"} - -<!--- TEST ARBITRARY_TIME -Completed 100000 actions in xxx ms -Counter = 100000 ---> - -It does not matter (for correctness) what context the actor itself is executed in. An actor is -a coroutine and a coroutine is executed sequentially, so confinement of the state to the specific coroutine -works as a solution to the problem of shared mutable state. Indeed, actors may modify their own private state, -but can only affect each other through messages (avoiding the need for any locks). - -Actor is more efficient than locking under load, because in this case it always has work to do and it does not -have to switch to a different context at all. - -> Note that an [actor] coroutine builder is a dual of [produce] coroutine builder. An actor is associated -> with the channel that it receives messages from, while a producer is associated with the channel that it -> sends elements to. -> -{type="note"} - <!--- MODULE kotlinx-coroutines-core --> <!--- INDEX kotlinx.coroutines --> [Dispatchers.Default]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/-dispatchers/-default.html [withContext]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/with-context.html -[CompletableDeferred]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/-completable-deferred/index.html <!--- INDEX kotlinx.coroutines.sync --> @@ -505,9 +382,4 @@ have to switch to a different context at all. [Mutex.unlock]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.sync/-mutex/unlock.html [withLock]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.sync/with-lock.html -<!--- INDEX kotlinx.coroutines.channels --> - -[actor]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.channels/actor.html -[produce]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.channels/produce.html - <!--- END --> diff --git a/dokka-templates/README.md b/dokka-templates/README.md new file mode 100644 index 00000000..08911777 --- /dev/null +++ b/dokka-templates/README.md @@ -0,0 +1,4 @@ +# Customize Dokka's HTML. +To customize Dokka's HTML output, place a file in this folder. +Dokka will find a template file there. If the file is not found, a default one will be used. +This folder is defined by the templatesDir property.
\ No newline at end of file diff --git a/gradle.properties b/gradle.properties index e452a07e..9e6dc023 100644 --- a/gradle.properties +++ b/gradle.properties @@ -3,34 +3,34 @@ # # Kotlin -version=1.6.4-SNAPSHOT +version=1.7.2-SNAPSHOT group=org.jetbrains.kotlinx -kotlin_version=1.6.21 +kotlin_version=1.8.20 # Dependencies junit_version=4.12 junit5_version=5.7.0 -atomicfu_version=0.17.3 +atomicfu_version=0.21.0 knit_version=0.4.0 html_version=0.7.2 -lincheck_version=2.14 -dokka_version=1.6.21 +lincheck_version=2.18.1 +dokka_version=1.8.10 byte_buddy_version=1.10.9 reactor_version=3.4.1 reactive_streams_version=1.0.3 rxjava2_version=2.2.8 rxjava3_version=3.0.2 -javafx_version=11.0.2 +javafx_version=17.0.2 javafx_plugin_version=0.0.8 -binary_compatibility_validator_version=0.11.0 -kover_version=0.5.0 -blockhound_version=1.0.2.RELEASE +binary_compatibility_validator_version=0.13.2 +kover_version=0.7.0-Beta +blockhound_version=1.0.8.RELEASE jna_version=5.9.0 # Android versions android_version=4.1.1.4 androidx_annotation_version=1.1.0 -robolectric_version=4.4 +robolectric_version=4.9 baksmali_version=2.2.7 # JS @@ -49,13 +49,13 @@ jsdom_global_version=3.0.2 kotlin.incremental.multiplatform=true kotlin.native.ignoreDisabledTargets=true -# Site generation -jekyll_version=4.0 - # JS IR backend sometimes crashes with out-of-memory # TODO: Remove once KT-37187 is fixed org.gradle.jvmargs=-Xmx3g kotlin.mpp.enableCompatibilityMetadataVariant=true kotlin.mpp.stability.nowarn=true -kotlinx.atomicfu.enableIrTransformation=true +kotlinx.atomicfu.enableJvmIrTransformation=true +# When the flag below is set to `true`, AtomicFU cannot process +# usages of `moveForward` in `ConcurrentLinkedList.kt` correctly. +kotlinx.atomicfu.enableJsIrTransformation=false diff --git a/gradle/compile-jvm-multiplatform.gradle b/gradle/compile-jvm-multiplatform.gradle index 88b71797..cb8325ca 100644 --- a/gradle/compile-jvm-multiplatform.gradle +++ b/gradle/compile-jvm-multiplatform.gradle @@ -10,6 +10,8 @@ kotlin { sourceSets { jvmMain.dependencies { compileOnly "org.codehaus.mojo:animal-sniffer-annotations:1.20" + // Workaround until https://github.com/JetBrains/kotlin/pull/4999 is picked up + api "org.jetbrains:annotations:23.0.0" } jvmTest.dependencies { diff --git a/gradle/compile-native-multiplatform.gradle b/gradle/compile-native-multiplatform.gradle index 0a247ede..3b275885 100644 --- a/gradle/compile-native-multiplatform.gradle +++ b/gradle/compile-native-multiplatform.gradle @@ -6,29 +6,45 @@ project.ext.nativeMainSets = [] project.ext.nativeTestSets = [] kotlin { - targets.metaClass.addTarget = { preset -> - def target = delegate.fromPreset(preset, preset.name) - project.ext.nativeMainSets.add(target.compilations['main'].kotlinSourceSets.first()) - project.ext.nativeTestSets.add(target.compilations['test'].kotlinSourceSets.first()) + targets { + delegate.metaClass.addTarget = { preset -> + def target = delegate.fromPreset(preset, preset.name) + project.ext.nativeMainSets.add(target.compilations['main'].kotlinSourceSets.first()) + project.ext.nativeTestSets.add(target.compilations['test'].kotlinSourceSets.first()) + } } targets { + // According to https://kotlinlang.org/docs/native-target-support.html + // Tier 1 addTarget(presets.linuxX64) - addTarget(presets.iosArm64) - addTarget(presets.iosArm32) - addTarget(presets.iosX64) addTarget(presets.macosX64) - addTarget(presets.mingwX64) - addTarget(presets.tvosArm64) - addTarget(presets.tvosX64) - addTarget(presets.watchosArm32) - addTarget(presets.watchosArm64) - addTarget(presets.watchosX86) - addTarget(presets.watchosX64) + addTarget(presets.macosArm64) addTarget(presets.iosSimulatorArm64) + addTarget(presets.iosX64) + + // Tier 2 + addTarget(presets.linuxArm64) addTarget(presets.watchosSimulatorArm64) + addTarget(presets.watchosX64) + addTarget(presets.watchosArm32) + addTarget(presets.watchosArm64) addTarget(presets.tvosSimulatorArm64) - addTarget(presets.macosArm64) + addTarget(presets.tvosX64) + addTarget(presets.tvosArm64) + addTarget(presets.iosArm64) + + // Tier 3 + addTarget(presets.androidNativeArm32) + addTarget(presets.androidNativeArm64) + addTarget(presets.androidNativeX86) + addTarget(presets.androidNativeX64) + addTarget(presets.mingwX64) + addTarget(presets.watchosDeviceArm64) + + // Deprecated, but were provided by coroutine; can be removed only when K/N drops the target + addTarget(presets.iosArm32) + addTarget(presets.watchosX86) } sourceSets { diff --git a/gradle/dokka.gradle.kts b/gradle/dokka.gradle.kts index 2470ded3..ba6956aa 100644 --- a/gradle/dokka.gradle.kts +++ b/gradle/dokka.gradle.kts @@ -27,6 +27,8 @@ tasks.withType(DokkaTaskPartial::class).configureEach { tasks.withType(DokkaTaskPartial::class).configureEach { suppressInheritedMembers.set(true) + pluginsMapConfiguration.set(mapOf("org.jetbrains.dokka.base.DokkaBase" to """{ "templatesDir" : "${rootProject.projectDir.toString().replace('\\', '/')}/dokka-templates" }""")) + dokkaSourceSets.configureEach { jdkVersion.set(11) includes.from("README.md") diff --git a/gradle/test-mocha-js.gradle b/gradle/test-mocha-js.gradle index 27d2e5b3..1ec297e4 100644 --- a/gradle/test-mocha-js.gradle +++ b/gradle/test-mocha-js.gradle @@ -23,13 +23,14 @@ def compileTestJsLegacy = tasks.hasProperty("compileTestKotlinJsLegacy") // todo: use atomicfu-transformed test files here (not critical) task testMochaNode(type: NodeTask, dependsOn: [compileTestJsLegacy, installDependenciesMochaNode]) { script = file("${node.nodeProjectDir.getAsFile().get()}/node_modules/mocha/bin/mocha") - args = [compileTestJsLegacy.outputFile.path, '--require', 'source-map-support/register'] + args = [compileTestJsLegacy.outputFileProperty.get().path, '--require', 'source-map-support/register'] if (project.hasProperty("teamcity")) args.addAll(['--reporter', 'mocha-teamcity-reporter']) } def jsLegacyTestTask = project.tasks.findByName('jsLegacyTest') ? jsLegacyTest : jsTest -jsLegacyTestTask.dependsOn testMochaNode +// TODO +//jsLegacyTestTask.dependsOn testMochaNode // -- Testing with Mocha under headless Chrome @@ -65,8 +66,8 @@ prepareMochaChrome.doLast { <script>mocha.setup('bdd');</script> <script src="$nodeProjDir/node_modules/kotlin/kotlin.js"></script> <script src="$nodeProjDir/node_modules/kotlin-test/kotlin-test.js"></script> - <script src="$compileJsLegacy.outputFile"></script> - <script src="$compileTestJsLegacy.outputFile"></script> + <script src="${compileJsLegacy.outputFileProperty.get().path}"></script> + <script src="${compileTestJsLegacy.outputFileProperty.get().path}"></script> <script>mocha.run();</script> </body> </html> @@ -75,7 +76,7 @@ prepareMochaChrome.doLast { task testMochaChrome(type: NodeTask, dependsOn: prepareMochaChrome) { script = file("${node.nodeProjectDir.getAsFile().get()}/node_modules/mocha-headless-chrome/bin/start") - args = [compileTestJsLegacy.outputFile.path, '--file', mochaChromeTestPage] + args = [compileTestJsLegacy.outputFileProperty.get().path, '--file', mochaChromeTestPage] if (project.hasProperty("teamcity")) args.addAll(['--reporter', 'mocha-teamcity-reporter']) } @@ -96,9 +97,9 @@ task installDependenciesMochaJsdom(type: NpmTask, dependsOn: [npmInstall]) { task testMochaJsdom(type: NodeTask, dependsOn: [compileTestJsLegacy, installDependenciesMochaJsdom]) { script = file("${node.nodeProjectDir.getAsFile().get()}/node_modules/mocha/bin/mocha") - args = [compileTestJsLegacy.outputFile.path, '--require', 'source-map-support/register', '--require', 'jsdom-global/register'] + args = [compileTestJsLegacy.outputFileProperty.get().path, '--require', 'source-map-support/register', '--require', 'jsdom-global/register'] if (project.hasProperty("teamcity")) args.addAll(['--reporter', 'mocha-teamcity-reporter']) } -jsLegacyTestTask.dependsOn testMochaJsdom - +// TODO +//jsLegacyTestTask.dependsOn testMochaJsdom diff --git a/integration-testing/README.md b/integration-testing/README.md index 0ede9b25..0218b23c 100644 --- a/integration-testing/README.md +++ b/integration-testing/README.md @@ -3,11 +3,13 @@ This is a supplementary project that provides integration tests. The tests are the following: -* `MavenPublicationValidator` depends on the published artifacts and tests artifacts binary content and absence of atomicfu in the classpath. -* `CoreAgentTest` checks that `kotlinx-coroutines-core` can be run as a Java agent. -* `DebugAgentTest` checks that the coroutine debugger can be run as a Java agent. -* `smokeTest` builds the test project that depends on coroutines. +* `mavenTest` depends on the published artifacts and tests artifacts binary content for absence of atomicfu in the classpath. +* `jvmCoreTest` miscellaneous tests that check the behaviour of `kotlinx-coroutines-core` dependency in a smoke manner. +* `coreAgentTest` checks that `kotlinx-coroutines-core` can be run as a Java agent. +* `debugAgentTest` checks that the coroutine debugger can be run as a Java agent. +* `debugDynamicAgentTest` checks that `kotlinx-coroutines-debug` agent can self-attach dynamically to JVM as a standalone dependency. +* `smokeTest` builds the multiplatform test project that depends on coroutines. The `integration-testing` project is expected to be in a subdirectory of the main `kotlinx.coroutines` project. -To run all the available tests: `cd integration-testing` + `./gradlew check`. +To run all the available tests: `./gradlew publishToMavenLocal` + `cd integration-testing` + `./gradlew check`. diff --git a/integration-testing/build.gradle b/integration-testing/build.gradle index 985a40ed..26ee9d99 100644 --- a/integration-testing/build.gradle +++ b/integration-testing/build.gradle @@ -2,13 +2,54 @@ * Copyright 2016-2021 JetBrains s.r.o. Use of this source code is governed by the Apache 2.0 license. */ -import org.jetbrains.kotlin.gradle.plugin.KotlinPlatformType + +buildscript { + + /* + * These property group is used to build kotlinx.coroutines against Kotlin compiler snapshot. + * How does it work: + * When build_snapshot_train is set to true, kotlin_version property is overridden with kotlin_snapshot_version, + * atomicfu_version is overwritten by TeamCity environment (AFU is built with snapshot and published to mavenLocal + * as previous step or the snapshot build). + * Additionally, mavenLocal and Sonatype snapshots are added to repository list and stress tests are disabled. + * DO NOT change the name of these properties without adapting kotlinx.train build chain. + */ + def prop = rootProject.properties['build_snapshot_train'] + ext.build_snapshot_train = prop != null && prop != "" + if (build_snapshot_train) { + ext.kotlin_version = rootProject.properties['kotlin_snapshot_version'] + if (kotlin_version == null) { + throw new IllegalArgumentException("'kotlin_snapshot_version' should be defined when building with snapshot compiler") + } + } + ext.native_targets_enabled = rootProject.properties['disable_native_targets'] == null + + // Determine if any project dependency is using a snapshot version + ext.using_snapshot_version = build_snapshot_train + rootProject.properties.each { key, value -> + if (key.endsWith("_version") && value instanceof String && value.endsWith("-SNAPSHOT")) { + println("NOTE: USING SNAPSHOT VERSION: $key=$value") + ext.using_snapshot_version = true + } + } + + if (using_snapshot_version) { + repositories { + mavenLocal() + maven { url "https://maven.pkg.jetbrains.space/kotlin/p/kotlin/dev" } + } + } + +} plugins { - id "org.jetbrains.kotlin.jvm" + id "org.jetbrains.kotlin.jvm" version "$kotlin_version" } repositories { + if (build_snapshot_train) { + maven { url "https://maven.pkg.jetbrains.space/kotlin/p/kotlin/dev" } + } mavenLocal() mavenCentral() } @@ -20,11 +61,13 @@ java { dependencies { testImplementation "org.jetbrains.kotlin:kotlin-test:$kotlin_version" + testImplementation "org.ow2.asm:asm:$asm_version" + implementation "org.jetbrains.kotlin:kotlin-stdlib-jdk8" } sourceSets { - // Test that relies on Guava to reflectively check all Throwable subclasses in coroutines - withGuavaTest { + // An assortment of tests for behavior of the core coroutines module on JVM + jvmCoreTest { kotlin compileClasspath += sourceSets.test.runtimeClasspath runtimeClasspath += sourceSets.test.runtimeClasspath @@ -57,7 +100,7 @@ sourceSets { } } - // Checks that kotlinx-coroutines-debug agent can self-attach dynamically to JVM as standalone dependency + // Checks that kotlinx-coroutines-debug agent can self-attach dynamically to JVM as a standalone dependency debugDynamicAgentTest { kotlin compileClasspath += sourceSets.test.runtimeClasspath @@ -87,9 +130,9 @@ compileDebugAgentTestKotlin { } } -task withGuavaTest(type: Test) { +task jvmCoreTest(type: Test) { environment "version", coroutines_version - def sourceSet = sourceSets.withGuavaTest + def sourceSet = sourceSets.jvmCoreTest testClassesDirs = sourceSet.output.classesDirs classpath = sourceSet.runtimeClasspath } @@ -129,5 +172,10 @@ compileTestKotlin { } check { - dependsOn([withGuavaTest, debugDynamicAgentTest, mavenTest, debugAgentTest, coreAgentTest, 'smokeTest:build']) + dependsOn([jvmCoreTest, debugDynamicAgentTest, mavenTest, debugAgentTest, coreAgentTest, 'smokeTest:build']) +} +compileKotlin { + kotlinOptions { + jvmTarget = "1.8" + } } diff --git a/integration-testing/gradle.properties b/integration-testing/gradle.properties index 1038d817..30b2b5ed 100644 --- a/integration-testing/gradle.properties +++ b/integration-testing/gradle.properties @@ -1,4 +1,6 @@ -kotlin_version=1.6.21 -coroutines_version=1.6.4-SNAPSHOT +kotlin_version=1.8.20 +coroutines_version=1.7.2-SNAPSHOT +asm_version=9.3 kotlin.code.style=official +kotlin.mpp.stability.nowarn=true diff --git a/integration-testing/settings.gradle b/integration-testing/settings.gradle index 67336c98..8584c05a 100644 --- a/integration-testing/settings.gradle +++ b/integration-testing/settings.gradle @@ -1,15 +1,8 @@ pluginManagement { - resolutionStrategy { - eachPlugin { - if (requested.id.id == "org.jetbrains.kotlin.multiplatform" || requested.id.id == "org.jetbrains.kotlin.jvm") { - useModule("org.jetbrains.kotlin:kotlin-gradle-plugin:$kotlin_version") - } - } - } - repositories { mavenCentral() maven { url "https://plugins.gradle.org/m2/" } + maven { url "https://maven.pkg.jetbrains.space/kotlin/p/kotlin/dev" } mavenLocal() } } diff --git a/integration-testing/smokeTest/build.gradle b/integration-testing/smokeTest/build.gradle index b200bb2f..26cd02b6 100644 --- a/integration-testing/smokeTest/build.gradle +++ b/integration-testing/smokeTest/build.gradle @@ -6,6 +6,7 @@ repositories { // Coroutines from the outer project are published by previous CI buils step mavenLocal() mavenCentral() + maven { url "https://maven.pkg.jetbrains.space/kotlin/p/kotlin/dev" } } kotlin { @@ -40,4 +41,12 @@ kotlin { } } } + targets { + configure([]) { + tasks.getByName(compilations.main.compileKotlinTaskName).kotlinOptions { + jvmTarget = "1.8" + } + } + } } + diff --git a/integration-testing/src/debugAgentTest/kotlin/PrecompiledDebugProbesTest.kt b/integration-testing/src/debugAgentTest/kotlin/PrecompiledDebugProbesTest.kt index 84886a18..ab207e09 100644 --- a/integration-testing/src/debugAgentTest/kotlin/PrecompiledDebugProbesTest.kt +++ b/integration-testing/src/debugAgentTest/kotlin/PrecompiledDebugProbesTest.kt @@ -16,10 +16,9 @@ class PrecompiledDebugProbesTest { @Test fun testClassFileContent() { val clz = Class.forName("kotlin.coroutines.jvm.internal.DebugProbesKt") - val className: String = clz.getName() - val classFileResourcePath = className.replace(".", "/") + ".class" - val stream = clz.classLoader.getResourceAsStream(classFileResourcePath)!! - val array = stream.readBytes() + val classFileResourcePath = clz.name.replace(".", "/") + ".class" + val array = clz.classLoader.getResourceAsStream(classFileResourcePath).use { it.readBytes() } + assertJava8Compliance(array) // we expect the integration testing project to be in a subdirectory of the main kotlinx.coroutines project val base = File("").absoluteFile.parentFile val probes = File(base, "kotlinx-coroutines-core/jvm/resources/DebugProbesKt.bin") @@ -31,8 +30,20 @@ class PrecompiledDebugProbesTest { assertTrue( array.contentEquals(binContent), "Compiled DebugProbesKt.class does not match the file shipped as a resource in kotlinx-coroutines-core. " + - "Typically it happens because of the Kotlin version update (-> binary metadata). In that case, run the same test with -Poverwrite.probes=true." + "Typically it happens because of the Kotlin version update (-> binary metadata). " + + "In that case, run the same test with -Poverwrite.probes=true." ) } } + + private fun assertJava8Compliance(classBytes: ByteArray) { + DataInputStream(classBytes.inputStream()).use { + val magic: Int = it.readInt() + if (magic != -0x35014542) throw IllegalArgumentException("Not a valid class!") + val minor: Int = it.readUnsignedShort() + val major: Int = it.readUnsignedShort() + assertEquals(52, major) + assertEquals(0, minor) + } + } } diff --git a/integration-testing/src/jvmCoreTest/kotlin/Jdk8InCoreIntegration.kt b/integration-testing/src/jvmCoreTest/kotlin/Jdk8InCoreIntegration.kt new file mode 100644 index 00000000..91eef7e2 --- /dev/null +++ b/integration-testing/src/jvmCoreTest/kotlin/Jdk8InCoreIntegration.kt @@ -0,0 +1,21 @@ +/* + * Copyright 2016-2022 JetBrains s.r.o. Use of this source code is governed by the Apache 2.0 license. + */ +package kotlinx.coroutines + +import kotlinx.coroutines.future.* +import org.junit.Test +import kotlin.test.* + +/* + * Integration test that ensures signatures from both the jdk8 and the core source sets of the kotlinx-coroutines-core subproject are used. + */ +class Jdk8InCoreIntegration { + + @Test + fun testFuture() = runBlocking<Unit> { + val future = future { yield(); 42 } + future.whenComplete { r, _ -> assertEquals(42, r) } + assertEquals(42, future.await()) + } +} diff --git a/integration-testing/src/withGuavaTest/kotlin/ListAllCoroutineThrowableSubclassesTest.kt b/integration-testing/src/jvmCoreTest/kotlin/ListAllCoroutineThrowableSubclassesTest.kt index fefcc005..7253658e 100644 --- a/integration-testing/src/withGuavaTest/kotlin/ListAllCoroutineThrowableSubclassesTest.kt +++ b/integration-testing/src/jvmCoreTest/kotlin/ListAllCoroutineThrowableSubclassesTest.kt @@ -7,6 +7,8 @@ package kotlinx.coroutines import com.google.common.reflect.* import kotlinx.coroutines.* import org.junit.Test +import java.io.Serializable +import java.lang.reflect.Modifier import kotlin.test.* class ListAllCoroutineThrowableSubclassesTest { @@ -25,19 +27,28 @@ class ListAllCoroutineThrowableSubclassesTest { "kotlinx.coroutines.JobCancellationException", "kotlinx.coroutines.internal.UndeliveredElementException", "kotlinx.coroutines.CompletionHandlerException", - "kotlinx.coroutines.DiagnosticCoroutineContextException", + "kotlinx.coroutines.internal.DiagnosticCoroutineContextException", + "kotlinx.coroutines.internal.ExceptionSuccessfullyProcessed", "kotlinx.coroutines.CoroutinesInternalError", "kotlinx.coroutines.channels.ClosedSendChannelException", "kotlinx.coroutines.channels.ClosedReceiveChannelException", "kotlinx.coroutines.flow.internal.ChildCancelledException", "kotlinx.coroutines.flow.internal.AbortFlowException", - ) + ) @Test fun testThrowableSubclassesAreSerializable() { val classes = ClassPath.from(this.javaClass.classLoader) .getTopLevelClassesRecursive("kotlinx.coroutines"); val throwables = classes.filter { Throwable::class.java.isAssignableFrom(it.load()) }.map { it.toString() } + for (throwable in throwables) { + for (field in throwable.javaClass.declaredFields) { + if (Modifier.isStatic(field.modifiers)) continue + val type = field.type + assertTrue(type.isPrimitive || Serializable::class.java.isAssignableFrom(type), + "Throwable $throwable has non-serializable field $field") + } + } assertEquals(knownThrowables.sorted(), throwables.sorted()) } } diff --git a/integration-testing/src/mavenTest/kotlin/MavenPublicationAtomicfuValidator.kt b/integration-testing/src/mavenTest/kotlin/MavenPublicationAtomicfuValidator.kt index dbb1921d..13bac015 100644 --- a/integration-testing/src/mavenTest/kotlin/MavenPublicationAtomicfuValidator.kt +++ b/integration-testing/src/mavenTest/kotlin/MavenPublicationAtomicfuValidator.kt @@ -4,12 +4,17 @@ package kotlinx.coroutines.validator -import org.junit.* -import org.junit.Assert.assertTrue +import org.junit.Test +import org.objectweb.asm.* +import org.objectweb.asm.ClassReader.* +import org.objectweb.asm.ClassWriter.* +import org.objectweb.asm.Opcodes.* import java.util.jar.* +import kotlin.test.* class MavenPublicationAtomicfuValidator { private val ATOMIC_FU_REF = "Lkotlinx/atomicfu/".toByteArray() + private val KOTLIN_METADATA_DESC = "Lkotlin/Metadata;" @Test fun testNoAtomicfuInClasspath() { @@ -34,19 +39,39 @@ class MavenPublicationAtomicfuValidator { for (e in entries()) { if (!e.name.endsWith(".class")) continue val bytes = getInputStream(e).use { it.readBytes() } - loop@for (i in 0 until bytes.size - ATOMIC_FU_REF.size) { - for (j in 0 until ATOMIC_FU_REF.size) { - if (bytes[i + j] != ATOMIC_FU_REF[j]) continue@loop - } + // The atomicfu compiler plugin does not remove atomic properties from metadata, + // so for now we check that there are no ATOMIC_FU_REF left in the class bytecode excluding metadata. + // This may be reverted after the fix in the compiler plugin transformer (for Kotlin 1.8.0). + val outBytes = bytes.eraseMetadata() + if (outBytes.checkBytes()) { foundClasses += e.name // report error at the end with all class names - break@loop } } if (foundClasses.isNotEmpty()) { error("Found references to atomicfu in jar file $name in the following class files: ${ - foundClasses.joinToString("") { "\n\t\t" + it } + foundClasses.joinToString("") { "\n\t\t" + it } }") } close() } + + private fun ByteArray.checkBytes(): Boolean { + loop@for (i in 0 until this.size - ATOMIC_FU_REF.size) { + for (j in 0 until ATOMIC_FU_REF.size) { + if (this[i + j] != ATOMIC_FU_REF[j]) continue@loop + } + return true + } + return false + } + + private fun ByteArray.eraseMetadata(): ByteArray { + val cw = ClassWriter(COMPUTE_MAXS or COMPUTE_FRAMES) + ClassReader(this).accept(object : ClassVisitor(ASM9, cw) { + override fun visitAnnotation(descriptor: String?, visible: Boolean): AnnotationVisitor? { + return if (descriptor == KOTLIN_METADATA_DESC) null else super.visitAnnotation(descriptor, visible) + } + }, SKIP_FRAMES) + return cw.toByteArray() + } } diff --git a/integration-testing/src/mavenTest/kotlin/MavenPublicationMetaInfValidator.kt b/integration-testing/src/mavenTest/kotlin/MavenPublicationMetaInfValidator.kt new file mode 100644 index 00000000..8ed2b823 --- /dev/null +++ b/integration-testing/src/mavenTest/kotlin/MavenPublicationMetaInfValidator.kt @@ -0,0 +1,70 @@ +/* + * Copyright 2016-2021 JetBrains s.r.o. Use of this source code is governed by the Apache 2.0 license. + */ + +package kotlinx.coroutines.validator + +import org.junit.Test +import org.objectweb.asm.* +import org.objectweb.asm.ClassReader.* +import org.objectweb.asm.ClassWriter.* +import org.objectweb.asm.Opcodes.* +import java.util.jar.* +import kotlin.test.* + +class MavenPublicationMetaInfValidator { + + @Test + fun testMetaInfCoreStructure() { + val clazz = Class.forName("kotlinx.coroutines.Job") + JarFile(clazz.protectionDomain.codeSource.location.file).checkMetaInfStructure( + setOf( + "MANIFEST.MF", + "kotlinx-coroutines-core.kotlin_module", + "com.android.tools/proguard/coroutines.pro", + "com.android.tools/r8/coroutines.pro", + "proguard/coroutines.pro", + "versions/9/module-info.class", + "kotlinx_coroutines_core.version" + ) + ) + } + + @Test + fun testMetaInfAndroidStructure() { + val clazz = Class.forName("kotlinx.coroutines.android.HandlerDispatcher") + JarFile(clazz.protectionDomain.codeSource.location.file).checkMetaInfStructure( + setOf( + "MANIFEST.MF", + "kotlinx-coroutines-android.kotlin_module", + "services/kotlinx.coroutines.CoroutineExceptionHandler", + "services/kotlinx.coroutines.internal.MainDispatcherFactory", + "com.android.tools/r8-from-1.6.0/coroutines.pro", + "com.android.tools/r8-upto-3.0.0/coroutines.pro", + "com.android.tools/proguard/coroutines.pro", + "proguard/coroutines.pro", + "versions/9/module-info.class", + "kotlinx_coroutines_android.version" + ) + ) + } + + private fun JarFile.checkMetaInfStructure(expected: Set<String>) { + val actual = HashSet<String>() + for (e in entries()) { + if (e.isDirectory() || !e.realName.contains("META-INF")) { + continue + } + val partialName = e.realName.substringAfter("META-INF/") + actual.add(partialName) + } + + if (actual != expected) { + val intersection = actual.intersect(expected) + val mismatch = actual.subtract(intersection) + expected.subtract(intersection) + fail("Mismatched files: " + mismatch) + } + + close() + } +} diff --git a/integration-testing/src/mavenTest/kotlin/MavenPublicationVersionValidator.kt b/integration-testing/src/mavenTest/kotlin/MavenPublicationVersionValidator.kt index da87d4cc..11529d2d 100644 --- a/integration-testing/src/mavenTest/kotlin/MavenPublicationVersionValidator.kt +++ b/integration-testing/src/mavenTest/kotlin/MavenPublicationVersionValidator.kt @@ -4,7 +4,6 @@ package kotlinx.coroutines.validator -import org.junit.* import org.junit.Test import java.util.jar.* import kotlin.test.* diff --git a/integration/README.md b/integration/README.md index 89100179..1d61ba4d 100644 --- a/integration/README.md +++ b/integration/README.md @@ -1,28 +1,10 @@ # Coroutines integration This directory contains modules that provide integration with various asynchronous callback- and future-based libraries. -Module name below corresponds to the artifact name in Maven/Gradle. +Module names below correspond to the artifact names in Maven/Gradle. ## Modules -* [kotlinx-coroutines-jdk8](kotlinx-coroutines-jdk8/README.md) -- integration with JDK8 `CompletableFuture` (Android API level 24). * [kotlinx-coroutines-guava](kotlinx-coroutines-guava/README.md) -- integration with Guava [ListenableFuture](https://github.com/google/guava/wiki/ListenableFutureExplained). * [kotlinx-coroutines-slf4j](kotlinx-coroutines-slf4j/README.md) -- integration with SLF4J [MDC](https://logback.qos.ch/manual/mdc.html). * [kotlinx-coroutines-play-services](kotlinx-coroutines-play-services) -- integration with Google Play Services [Tasks API](https://developers.google.com/android/guides/tasks). - -## Contributing - -Follow the following simple guidelines when contributing integration with your favorite library: - -* Keep it simple and general. Ideally it should fit into a single file. If it does not fit, then consider - a separate GitHub project to host this integration. -* Follow the example of other modules. - Cut-and-paste [kotlinx-coroutines-guava](kotlinx-coroutines-guava) module as a template. -* Write tests and documentation, include top-level `README.md` with short overview and example. -* Reference the new module from all the places: - * List of modules in this document. - * List of modules in top-level [`settings.gradle`](../settings.gradle). - * List of modules at the root of documentation site in [`site/docs/index.md`](../site/docs/index.md). - * List of integrations in the root [README.md](../README.md). -* Update links to documentation website as explained [here](../knit/README.md#usage). -* Squash your contribution to a single commit and create pull request to `develop` branch. diff --git a/integration/kotlinx-coroutines-guava/src/module-info.java b/integration/kotlinx-coroutines-guava/src/module-info.java new file mode 100644 index 00000000..0b8ccafd --- /dev/null +++ b/integration/kotlinx-coroutines-guava/src/module-info.java @@ -0,0 +1,7 @@ +module kotlinx.coroutines.guava { + requires kotlin.stdlib; + requires kotlinx.coroutines.core; + requires com.google.common; + + exports kotlinx.coroutines.guava; +} diff --git a/integration/kotlinx-coroutines-jdk8/README.md b/integration/kotlinx-coroutines-jdk8/README.md index 321e2934..56e145fc 100644 --- a/integration/kotlinx-coroutines-jdk8/README.md +++ b/integration/kotlinx-coroutines-jdk8/README.md @@ -1,68 +1,3 @@ -# Module kotlinx-coroutines-jdk8 +# Stub module -Integration with JDK8 [CompletableFuture] (Android API level 24). - -Coroutine builders: - -| **Name** | **Result** | **Scope** | **Description** -| -------- | ------------------- | ---------------- | --------------- -| [future] | [CompletableFuture] | [CoroutineScope] | Returns a single value with the future result - -Extension functions: - -| **Name** | **Description** -| -------- | --------------- -| [CompletionStage.await][java.util.concurrent.CompletionStage.await] | Awaits for completion of the completion stage -| [CompletionStage.asDeferred][java.util.concurrent.CompletionStage.asDeferred] | Converts completion stage to an instance of [Deferred] -| [Deferred.asCompletableFuture][kotlinx.coroutines.Deferred.asCompletableFuture] | Converts a deferred value to the future - -## Example - -Given the following functions defined in some Java API: - -```java -public CompletableFuture<Image> loadImageAsync(String name); // starts async image loading -public Image combineImages(Image image1, Image image2); // synchronously combines two images using some algorithm -``` - -We can consume this API from Kotlin coroutine to load two images and combine then asynchronously. -The resulting function returns `CompletableFuture<Image>` for ease of use back from Java. - -```kotlin -fun combineImagesAsync(name1: String, name2: String): CompletableFuture<Image> = future { - val future1 = loadImageAsync(name1) // start loading first image - val future2 = loadImageAsync(name2) // start loading second image - combineImages(future1.await(), future2.await()) // wait for both, combine, and return result -} -``` - -Note that this module should be used only for integration with existing Java APIs based on `CompletableFuture`. -Writing pure-Kotlin code that uses `CompletableFuture` is highly not recommended, since the resulting APIs based -on the futures are quite error-prone. See the discussion on -[Asynchronous Programming Styles](https://github.com/Kotlin/KEEP/blob/master/proposals/coroutines.md#asynchronous-programming-styles) -for details on general problems pertaining to any future-based API and keep in mind that `CompletableFuture` exposes -a _blocking_ method -[get](https://docs.oracle.com/javase/8/docs/api/java/util/concurrent/Future.html#get--) -that makes it especially bad choice for coroutine-based Kotlin code. - -# Package kotlinx.coroutines.future - -Integration with JDK8 [CompletableFuture] (Android API level 24). - -[CompletableFuture]: https://docs.oracle.com/javase/8/docs/api/java/util/concurrent/CompletableFuture.html - -<!--- MODULE kotlinx-coroutines-core --> -<!--- INDEX kotlinx.coroutines --> - -[CoroutineScope]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/-coroutine-scope/index.html -[Deferred]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/-deferred/index.html - -<!--- MODULE kotlinx-coroutines-jdk8 --> -<!--- INDEX kotlinx.coroutines.future --> - -[future]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-jdk8/kotlinx.coroutines.future/future.html -[java.util.concurrent.CompletionStage.await]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-jdk8/kotlinx.coroutines.future/await.html -[java.util.concurrent.CompletionStage.asDeferred]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-jdk8/kotlinx.coroutines.future/as-deferred.html -[kotlinx.coroutines.Deferred.asCompletableFuture]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-jdk8/kotlinx.coroutines.future/as-completable-future.html - -<!--- END --> +Stub module for backwards compatibility. Since 1.7.0, this module was merged with core. diff --git a/integration/kotlinx-coroutines-jdk8/api/kotlinx-coroutines-jdk8.api b/integration/kotlinx-coroutines-jdk8/api/kotlinx-coroutines-jdk8.api index 4ee57845..e69de29b 100644 --- a/integration/kotlinx-coroutines-jdk8/api/kotlinx-coroutines-jdk8.api +++ b/integration/kotlinx-coroutines-jdk8/api/kotlinx-coroutines-jdk8.api @@ -1,22 +0,0 @@ -public final class kotlinx/coroutines/future/FutureKt { - public static final fun asCompletableFuture (Lkotlinx/coroutines/Deferred;)Ljava/util/concurrent/CompletableFuture; - public static final fun asCompletableFuture (Lkotlinx/coroutines/Job;)Ljava/util/concurrent/CompletableFuture; - public static final fun asDeferred (Ljava/util/concurrent/CompletionStage;)Lkotlinx/coroutines/Deferred; - public static final fun await (Ljava/util/concurrent/CompletionStage;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; - public static final fun future (Lkotlinx/coroutines/CoroutineScope;Lkotlin/coroutines/CoroutineContext;Lkotlinx/coroutines/CoroutineStart;Lkotlin/jvm/functions/Function2;)Ljava/util/concurrent/CompletableFuture; - public static synthetic fun future$default (Lkotlinx/coroutines/CoroutineScope;Lkotlin/coroutines/CoroutineContext;Lkotlinx/coroutines/CoroutineStart;Lkotlin/jvm/functions/Function2;ILjava/lang/Object;)Ljava/util/concurrent/CompletableFuture; -} - -public final class kotlinx/coroutines/stream/StreamKt { - public static final fun consumeAsFlow (Ljava/util/stream/Stream;)Lkotlinx/coroutines/flow/Flow; -} - -public final class kotlinx/coroutines/time/TimeKt { - public static final fun debounce (Lkotlinx/coroutines/flow/Flow;Ljava/time/Duration;)Lkotlinx/coroutines/flow/Flow; - public static final fun delay (Ljava/time/Duration;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; - public static final fun onTimeout (Lkotlinx/coroutines/selects/SelectBuilder;Ljava/time/Duration;Lkotlin/jvm/functions/Function1;)V - public static final fun sample (Lkotlinx/coroutines/flow/Flow;Ljava/time/Duration;)Lkotlinx/coroutines/flow/Flow; - public static final fun withTimeout (Ljava/time/Duration;Lkotlin/jvm/functions/Function2;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; - public static final fun withTimeoutOrNull (Ljava/time/Duration;Lkotlin/jvm/functions/Function2;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; -} - diff --git a/integration/kotlinx-coroutines-jdk8/src/module-info.java b/integration/kotlinx-coroutines-jdk8/src/module-info.java new file mode 100644 index 00000000..d83596c2 --- /dev/null +++ b/integration/kotlinx-coroutines-jdk8/src/module-info.java @@ -0,0 +1,3 @@ +@SuppressWarnings("JavaModuleNaming") +module kotlinx.coroutines.jdk8 { +} diff --git a/integration/kotlinx-coroutines-slf4j/src/module-info.java b/integration/kotlinx-coroutines-slf4j/src/module-info.java new file mode 100644 index 00000000..57e5aae4 --- /dev/null +++ b/integration/kotlinx-coroutines-slf4j/src/module-info.java @@ -0,0 +1,7 @@ +module kotlinx.coroutines.slf4j { + requires kotlin.stdlib; + requires kotlinx.coroutines.core; + requires org.slf4j; + + exports kotlinx.coroutines.slf4j; +} diff --git a/js/example-frontend-js/build.gradle.kts b/js/example-frontend-js/build.gradle.kts index a78ac3dc..1cc587b7 100644 --- a/js/example-frontend-js/build.gradle.kts +++ b/js/example-frontend-js/build.gradle.kts @@ -10,7 +10,9 @@ kotlin { directory = directory.parentFile.resolve("dist") } commonWebpackConfig { - cssSupport.enabled = true + cssSupport { + enabled.set(true) + } } testTask { useKarma { diff --git a/kotlinx-coroutines-core/api/kotlinx-coroutines-core.api b/kotlinx-coroutines-core/api/kotlinx-coroutines-core.api index dd7f889e..9b548ac4 100644 --- a/kotlinx-coroutines-core/api/kotlinx-coroutines-core.api +++ b/kotlinx-coroutines-core/api/kotlinx-coroutines-core.api @@ -51,7 +51,7 @@ public final class kotlinx/coroutines/CancellableContinuation$DefaultImpls { public static synthetic fun tryResume$default (Lkotlinx/coroutines/CancellableContinuation;Ljava/lang/Object;Ljava/lang/Object;ILjava/lang/Object;)Ljava/lang/Object; } -public class kotlinx/coroutines/CancellableContinuationImpl : kotlin/coroutines/jvm/internal/CoroutineStackFrame, kotlinx/coroutines/CancellableContinuation { +public class kotlinx/coroutines/CancellableContinuationImpl : kotlinx/coroutines/DispatchedTask, kotlin/coroutines/jvm/internal/CoroutineStackFrame, kotlinx/coroutines/CancellableContinuation, kotlinx/coroutines/Waiter { public fun <init> (Lkotlin/coroutines/Continuation;I)V public final fun callCancelHandler (Lkotlinx/coroutines/CancelHandler;Ljava/lang/Throwable;)V public final fun callOnCancellation (Lkotlin/jvm/functions/Function1;Ljava/lang/Throwable;)V @@ -64,6 +64,7 @@ public class kotlinx/coroutines/CancellableContinuationImpl : kotlin/coroutines/ public fun getStackTraceElement ()Ljava/lang/StackTraceElement; public fun initCancellability ()V public fun invokeOnCancellation (Lkotlin/jvm/functions/Function1;)V + public fun invokeOnCancellation (Lkotlinx/coroutines/internal/Segment;I)V public fun isActive ()Z public fun isCancelled ()Z public fun isCompleted ()Z @@ -83,6 +84,13 @@ public final class kotlinx/coroutines/CancellableContinuationKt { public static final fun suspendCancellableCoroutine (Lkotlin/jvm/functions/Function1;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; } +public final class kotlinx/coroutines/ChildContinuation { + public final field child Lkotlinx/coroutines/CancellableContinuationImpl; + public fun <init> (Lkotlinx/coroutines/CancellableContinuationImpl;)V + public synthetic fun invoke (Ljava/lang/Object;)Ljava/lang/Object; + public fun invoke (Ljava/lang/Throwable;)V +} + public abstract interface class kotlinx/coroutines/ChildHandle : kotlinx/coroutines/DisposableHandle { public abstract fun childCancelled (Ljava/lang/Throwable;)Z public abstract fun getParent ()Lkotlinx/coroutines/Job; @@ -199,6 +207,25 @@ public final class kotlinx/coroutines/CoroutineExceptionHandlerKt { public static final fun handleCoroutineException (Lkotlin/coroutines/CoroutineContext;Ljava/lang/Throwable;)V } +public final class kotlinx/coroutines/CoroutineId : kotlin/coroutines/AbstractCoroutineContextElement, kotlinx/coroutines/ThreadContextElement { + public static final field Key Lkotlinx/coroutines/CoroutineId$Key; + public fun <init> (J)V + public final fun component1 ()J + public final fun copy (J)Lkotlinx/coroutines/CoroutineId; + public static synthetic fun copy$default (Lkotlinx/coroutines/CoroutineId;JILjava/lang/Object;)Lkotlinx/coroutines/CoroutineId; + public fun equals (Ljava/lang/Object;)Z + public final fun getId ()J + public fun hashCode ()I + public synthetic fun restoreThreadContext (Lkotlin/coroutines/CoroutineContext;Ljava/lang/Object;)V + public fun restoreThreadContext (Lkotlin/coroutines/CoroutineContext;Ljava/lang/String;)V + public fun toString ()Ljava/lang/String; + public synthetic fun updateThreadContext (Lkotlin/coroutines/CoroutineContext;)Ljava/lang/Object; + public fun updateThreadContext (Lkotlin/coroutines/CoroutineContext;)Ljava/lang/String; +} + +public final class kotlinx/coroutines/CoroutineId$Key : kotlin/coroutines/CoroutineContext$Key { +} + public final class kotlinx/coroutines/CoroutineName : kotlin/coroutines/AbstractCoroutineContextElement { public static final field Key Lkotlinx/coroutines/CoroutineName$Key; public fun <init> (Ljava/lang/String;)V @@ -287,6 +314,15 @@ public final class kotlinx/coroutines/DelayKt { public abstract interface annotation class kotlinx/coroutines/DelicateCoroutinesApi : java/lang/annotation/Annotation { } +public final class kotlinx/coroutines/DispatchedCoroutine { + public static final fun get_decision$FU ()Ljava/util/concurrent/atomic/AtomicIntegerFieldUpdater; +} + +public abstract class kotlinx/coroutines/DispatchedTask : kotlinx/coroutines/scheduling/Task { + public field resumeMode I + public final fun run ()V +} + public final class kotlinx/coroutines/DispatchedTaskKt { public static final field MODE_CANCELLABLE I } @@ -302,6 +338,7 @@ public final class kotlinx/coroutines/Dispatchers { public final class kotlinx/coroutines/DispatchersKt { public static final field IO_PARALLELISM_PROPERTY_NAME Ljava/lang/String; + public static final synthetic fun getIO (Lkotlinx/coroutines/Dispatchers;)Lkotlinx/coroutines/CoroutineDispatcher; } public abstract interface class kotlinx/coroutines/DisposableHandle { @@ -309,7 +346,9 @@ public abstract interface class kotlinx/coroutines/DisposableHandle { } public final class kotlinx/coroutines/EventLoopKt { + public static final fun isIoDispatcherThread (Ljava/lang/Thread;)Z public static final fun processNextEventInCurrentThread ()J + public static final fun runSingleTaskFromCurrentSystemDispatcher ()J } public final class kotlinx/coroutines/ExceptionsKt { @@ -360,6 +399,7 @@ public abstract interface class kotlinx/coroutines/Job : kotlin/coroutines/Corou public abstract fun getCancellationException ()Ljava/util/concurrent/CancellationException; public abstract fun getChildren ()Lkotlin/sequences/Sequence; public abstract fun getOnJoin ()Lkotlinx/coroutines/selects/SelectClause0; + public abstract fun getParent ()Lkotlinx/coroutines/Job; public abstract fun invokeOnCompletion (Lkotlin/jvm/functions/Function1;)Lkotlinx/coroutines/DisposableHandle; public abstract fun invokeOnCompletion (ZZLkotlin/jvm/functions/Function1;)Lkotlinx/coroutines/DisposableHandle; public abstract fun isActive ()Z @@ -422,10 +462,11 @@ public final class kotlinx/coroutines/JobKt { public static final fun isActive (Lkotlin/coroutines/CoroutineContext;)Z } -public class kotlinx/coroutines/JobSupport : kotlinx/coroutines/ChildJob, kotlinx/coroutines/Job, kotlinx/coroutines/ParentJob, kotlinx/coroutines/selects/SelectClause0 { +public class kotlinx/coroutines/JobSupport : kotlinx/coroutines/ChildJob, kotlinx/coroutines/Job, kotlinx/coroutines/ParentJob { public fun <init> (Z)V protected fun afterCompletion (Ljava/lang/Object;)V public final fun attachChild (Lkotlinx/coroutines/ChildJob;)Lkotlinx/coroutines/ChildHandle; + protected final fun awaitInternal (Lkotlin/coroutines/Continuation;)Ljava/lang/Object; public synthetic fun cancel ()V public synthetic fun cancel (Ljava/lang/Throwable;)Z public fun cancel (Ljava/util/concurrent/CancellationException;)V @@ -442,7 +483,9 @@ public class kotlinx/coroutines/JobSupport : kotlinx/coroutines/ChildJob, kotlin protected final fun getCompletionCauseHandled ()Z public final fun getCompletionExceptionOrNull ()Ljava/lang/Throwable; public final fun getKey ()Lkotlin/coroutines/CoroutineContext$Key; + protected final fun getOnAwaitInternal ()Lkotlinx/coroutines/selects/SelectClause1; public final fun getOnJoin ()Lkotlinx/coroutines/selects/SelectClause0; + public fun getParent ()Lkotlinx/coroutines/Job; protected fun handleJobException (Ljava/lang/Throwable;)Z protected final fun initParentJob (Lkotlinx/coroutines/Job;)V public final fun invokeOnCompletion (Lkotlin/jvm/functions/Function1;)Lkotlinx/coroutines/DisposableHandle; @@ -460,7 +503,6 @@ public class kotlinx/coroutines/JobSupport : kotlinx/coroutines/ChildJob, kotlin public final fun parentCancelled (Lkotlinx/coroutines/ParentJob;)V public fun plus (Lkotlin/coroutines/CoroutineContext;)Lkotlin/coroutines/CoroutineContext; public fun plus (Lkotlinx/coroutines/Job;)Lkotlinx/coroutines/Job; - public final fun registerSelectClause0 (Lkotlinx/coroutines/selects/SelectInstance;Lkotlin/jvm/functions/Function1;)V public final fun start ()Z protected final fun toCancellationException (Ljava/lang/Throwable;Ljava/lang/String;)Ljava/util/concurrent/CancellationException; public static synthetic fun toCancellationException$default (Lkotlinx/coroutines/JobSupport;Ljava/lang/Throwable;Ljava/lang/String;ILjava/lang/Object;)Ljava/util/concurrent/CancellationException; @@ -485,6 +527,7 @@ public final class kotlinx/coroutines/NonCancellable : kotlin/coroutines/Abstrac public fun getCancellationException ()Ljava/util/concurrent/CancellationException; public fun getChildren ()Lkotlin/sequences/Sequence; public fun getOnJoin ()Lkotlinx/coroutines/selects/SelectClause0; + public fun getParent ()Lkotlinx/coroutines/Job; public fun invokeOnCompletion (Lkotlin/jvm/functions/Function1;)Lkotlinx/coroutines/DisposableHandle; public fun invokeOnCompletion (ZZLkotlin/jvm/functions/Function1;)Lkotlinx/coroutines/DisposableHandle; public fun isActive ()Z @@ -747,10 +790,10 @@ public final class kotlinx/coroutines/channels/ChannelsKt { public static final synthetic fun maxWith (Lkotlinx/coroutines/channels/ReceiveChannel;Ljava/util/Comparator;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; public static final synthetic fun minWith (Lkotlinx/coroutines/channels/ReceiveChannel;Ljava/util/Comparator;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; public static final synthetic fun none (Lkotlinx/coroutines/channels/ReceiveChannel;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; - public static final fun onReceiveOrNull (Lkotlinx/coroutines/channels/ReceiveChannel;)Lkotlinx/coroutines/selects/SelectClause1; - public static final fun receiveOrNull (Lkotlinx/coroutines/channels/ReceiveChannel;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; + public static final synthetic fun onReceiveOrNull (Lkotlinx/coroutines/channels/ReceiveChannel;)Lkotlinx/coroutines/selects/SelectClause1; + public static final synthetic fun receiveOrNull (Lkotlinx/coroutines/channels/ReceiveChannel;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; public static final synthetic fun requireNoNulls (Lkotlinx/coroutines/channels/ReceiveChannel;)Lkotlinx/coroutines/channels/ReceiveChannel; - public static final fun sendBlocking (Lkotlinx/coroutines/channels/SendChannel;Ljava/lang/Object;)V + public static final synthetic fun sendBlocking (Lkotlinx/coroutines/channels/SendChannel;Ljava/lang/Object;)V public static final synthetic fun single (Lkotlinx/coroutines/channels/ReceiveChannel;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; public static final synthetic fun singleOrNull (Lkotlinx/coroutines/channels/ReceiveChannel;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; public static final synthetic fun take (Lkotlinx/coroutines/channels/ReceiveChannel;ILkotlin/coroutines/CoroutineContext;)Lkotlinx/coroutines/channels/ReceiveChannel; @@ -869,9 +912,7 @@ public final class kotlinx/coroutines/channels/TickerMode : java/lang/Enum { } public final class kotlinx/coroutines/debug/internal/DebugCoroutineInfo { - public fun <init> (Lkotlinx/coroutines/debug/internal/DebugCoroutineInfoImpl;Lkotlin/coroutines/CoroutineContext;)V public final fun getContext ()Lkotlin/coroutines/CoroutineContext; - public final fun getCreationStackBottom ()Lkotlin/coroutines/jvm/internal/CoroutineStackFrame; public final fun getCreationStackTrace ()Ljava/util/List; public final fun getLastObservedFrame ()Lkotlin/coroutines/jvm/internal/CoroutineStackFrame; public final fun getLastObservedThread ()Ljava/lang/Thread; @@ -880,6 +921,37 @@ public final class kotlinx/coroutines/debug/internal/DebugCoroutineInfo { public final fun lastObservedStackTrace ()Ljava/util/List; } +public final class kotlinx/coroutines/debug/internal/DebugCoroutineInfoImpl { + public field _lastObservedFrame Ljava/lang/ref/WeakReference; + public field _state Ljava/lang/String; + public field lastObservedThread Ljava/lang/Thread; + public final field sequenceNumber J + public final fun getContext ()Lkotlin/coroutines/CoroutineContext; + public final fun getCreationStackTrace ()Ljava/util/List; + public fun toString ()Ljava/lang/String; +} + +public final class kotlinx/coroutines/debug/internal/DebugProbesImpl { + public static final field INSTANCE Lkotlinx/coroutines/debug/internal/DebugProbesImpl; + public final fun dumpCoroutinesInfo ()Ljava/util/List; + public final fun dumpCoroutinesInfoAsJsonAndReferences ()[Ljava/lang/Object; + public final fun dumpDebuggerInfo ()Ljava/util/List; + public final fun enhanceStackTraceWithThreadDump (Lkotlinx/coroutines/debug/internal/DebugCoroutineInfo;Ljava/util/List;)Ljava/util/List; + public final fun enhanceStackTraceWithThreadDumpAsJson (Lkotlinx/coroutines/debug/internal/DebugCoroutineInfo;)Ljava/lang/String; + public final fun getIgnoreCoroutinesWithEmptyContext ()Z + public final fun isInstalled ()Z + public final fun setIgnoreCoroutinesWithEmptyContext (Z)V +} + +public final class kotlinx/coroutines/debug/internal/DebugProbesImpl$CoroutineOwner : kotlin/coroutines/Continuation, kotlin/coroutines/jvm/internal/CoroutineStackFrame { + public final field info Lkotlinx/coroutines/debug/internal/DebugCoroutineInfoImpl; + public fun getCallerFrame ()Lkotlin/coroutines/jvm/internal/CoroutineStackFrame; + public fun getContext ()Lkotlin/coroutines/CoroutineContext; + public fun getStackTraceElement ()Ljava/lang/StackTraceElement; + public fun resumeWith (Ljava/lang/Object;)V + public fun toString ()Ljava/lang/String; +} + public final class kotlinx/coroutines/debug/internal/DebuggerInfo : java/io/Serializable { public fun <init> (Lkotlinx/coroutines/debug/internal/DebugCoroutineInfoImpl;Lkotlin/coroutines/CoroutineContext;)V public final fun getCoroutineId ()Ljava/lang/Long; @@ -892,6 +964,12 @@ public final class kotlinx/coroutines/debug/internal/DebuggerInfo : java/io/Seri public final fun getState ()Ljava/lang/String; } +public final class kotlinx/coroutines/debug/internal/StackTraceFrame : kotlin/coroutines/jvm/internal/CoroutineStackFrame { + public final field stackTraceElement Ljava/lang/StackTraceElement; + public fun getCallerFrame ()Lkotlin/coroutines/jvm/internal/CoroutineStackFrame; + public fun getStackTraceElement ()Ljava/lang/StackTraceElement; +} + public abstract class kotlinx/coroutines/flow/AbstractFlow : kotlinx/coroutines/flow/CancellableFlow, kotlinx/coroutines/flow/Flow { public fun <init> ()V public final fun collect (Lkotlinx/coroutines/flow/FlowCollector;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; @@ -969,6 +1047,7 @@ public final class kotlinx/coroutines/flow/FlowKt { public static final fun emitAll (Lkotlinx/coroutines/flow/FlowCollector;Lkotlinx/coroutines/flow/Flow;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; public static final fun emptyFlow ()Lkotlinx/coroutines/flow/Flow; public static final fun filter (Lkotlinx/coroutines/flow/Flow;Lkotlin/jvm/functions/Function2;)Lkotlinx/coroutines/flow/Flow; + public static final fun filterIsInstance (Lkotlinx/coroutines/flow/Flow;Lkotlin/reflect/KClass;)Lkotlinx/coroutines/flow/Flow; public static final fun filterNot (Lkotlinx/coroutines/flow/Flow;Lkotlin/jvm/functions/Function2;)Lkotlinx/coroutines/flow/Flow; public static final fun filterNotNull (Lkotlinx/coroutines/flow/Flow;)Lkotlinx/coroutines/flow/Flow; public static final fun first (Lkotlinx/coroutines/flow/Flow;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; @@ -1047,6 +1126,7 @@ public final class kotlinx/coroutines/flow/FlowKt { public static final fun switchMap (Lkotlinx/coroutines/flow/Flow;Lkotlin/jvm/functions/Function2;)Lkotlinx/coroutines/flow/Flow; public static final fun take (Lkotlinx/coroutines/flow/Flow;I)Lkotlinx/coroutines/flow/Flow; public static final fun takeWhile (Lkotlinx/coroutines/flow/Flow;Lkotlin/jvm/functions/Function2;)Lkotlinx/coroutines/flow/Flow; + public static final fun timeout-HG0u8IE (Lkotlinx/coroutines/flow/Flow;J)Lkotlinx/coroutines/flow/Flow; public static final fun toCollection (Lkotlinx/coroutines/flow/Flow;Ljava/util/Collection;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; public static final fun toList (Lkotlinx/coroutines/flow/Flow;Ljava/util/List;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; public static synthetic fun toList$default (Lkotlinx/coroutines/flow/Flow;Ljava/util/List;Lkotlin/coroutines/Continuation;ILjava/lang/Object;)Ljava/lang/Object; @@ -1170,6 +1250,15 @@ public final class kotlinx/coroutines/flow/internal/SendingCollector : kotlinx/c public fun emit (Ljava/lang/Object;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; } +public final class kotlinx/coroutines/future/FutureKt { + public static final fun asCompletableFuture (Lkotlinx/coroutines/Deferred;)Ljava/util/concurrent/CompletableFuture; + public static final fun asCompletableFuture (Lkotlinx/coroutines/Job;)Ljava/util/concurrent/CompletableFuture; + public static final fun asDeferred (Ljava/util/concurrent/CompletionStage;)Lkotlinx/coroutines/Deferred; + public static final fun await (Ljava/util/concurrent/CompletionStage;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; + public static final fun future (Lkotlinx/coroutines/CoroutineScope;Lkotlin/coroutines/CoroutineContext;Lkotlinx/coroutines/CoroutineStart;Lkotlin/jvm/functions/Function2;)Ljava/util/concurrent/CompletableFuture; + public static synthetic fun future$default (Lkotlinx/coroutines/CoroutineScope;Lkotlin/coroutines/CoroutineContext;Lkotlinx/coroutines/CoroutineStart;Lkotlin/jvm/functions/Function2;ILjava/lang/Object;)Ljava/util/concurrent/CompletableFuture; +} + public final class kotlinx/coroutines/intrinsics/CancellableKt { public static final fun startCoroutineCancellable (Lkotlin/jvm/functions/Function1;Lkotlin/coroutines/Continuation;)V } @@ -1191,6 +1280,15 @@ public class kotlinx/coroutines/scheduling/ExperimentalCoroutineDispatcher : kot public fun toString ()Ljava/lang/String; } +public abstract class kotlinx/coroutines/scheduling/Task : java/lang/Runnable { + public field submissionTime J +} + +public final class kotlinx/coroutines/selects/OnTimeoutKt { + public static final fun onTimeout (Lkotlinx/coroutines/selects/SelectBuilder;JLkotlin/jvm/functions/Function1;)V + public static final fun onTimeout-8Mi8wO0 (Lkotlinx/coroutines/selects/SelectBuilder;JLkotlin/jvm/functions/Function1;)V +} + public abstract interface class kotlinx/coroutines/selects/SelectBuilder { public abstract fun invoke (Lkotlinx/coroutines/selects/SelectClause0;Lkotlin/jvm/functions/Function1;)V public abstract fun invoke (Lkotlinx/coroutines/selects/SelectClause1;Lkotlin/jvm/functions/Function2;)V @@ -1201,79 +1299,91 @@ public abstract interface class kotlinx/coroutines/selects/SelectBuilder { public final class kotlinx/coroutines/selects/SelectBuilder$DefaultImpls { public static fun invoke (Lkotlinx/coroutines/selects/SelectBuilder;Lkotlinx/coroutines/selects/SelectClause2;Lkotlin/jvm/functions/Function2;)V + public static fun onTimeout (Lkotlinx/coroutines/selects/SelectBuilder;JLkotlin/jvm/functions/Function1;)V } -public final class kotlinx/coroutines/selects/SelectBuilderImpl : kotlinx/coroutines/internal/LockFreeLinkedListHead, kotlin/coroutines/Continuation, kotlin/coroutines/jvm/internal/CoroutineStackFrame, kotlinx/coroutines/selects/SelectBuilder, kotlinx/coroutines/selects/SelectInstance { +public final class kotlinx/coroutines/selects/SelectBuilderImpl : kotlinx/coroutines/selects/SelectImplementation { public fun <init> (Lkotlin/coroutines/Continuation;)V - public fun disposeOnSelect (Lkotlinx/coroutines/DisposableHandle;)V - public fun getCallerFrame ()Lkotlin/coroutines/jvm/internal/CoroutineStackFrame; - public fun getCompletion ()Lkotlin/coroutines/Continuation; - public fun getContext ()Lkotlin/coroutines/CoroutineContext; public final fun getResult ()Ljava/lang/Object; - public fun getStackTraceElement ()Ljava/lang/StackTraceElement; public final fun handleBuilderException (Ljava/lang/Throwable;)V - public fun invoke (Lkotlinx/coroutines/selects/SelectClause0;Lkotlin/jvm/functions/Function1;)V - public fun invoke (Lkotlinx/coroutines/selects/SelectClause1;Lkotlin/jvm/functions/Function2;)V - public fun invoke (Lkotlinx/coroutines/selects/SelectClause2;Ljava/lang/Object;Lkotlin/jvm/functions/Function2;)V - public fun invoke (Lkotlinx/coroutines/selects/SelectClause2;Lkotlin/jvm/functions/Function2;)V - public fun isSelected ()Z - public fun onTimeout (JLkotlin/jvm/functions/Function1;)V - public fun performAtomicTrySelect (Lkotlinx/coroutines/internal/AtomicDesc;)Ljava/lang/Object; - public fun resumeSelectWithException (Ljava/lang/Throwable;)V - public fun resumeWith (Ljava/lang/Object;)V - public fun toString ()Ljava/lang/String; - public fun trySelect ()Z - public fun trySelectOther (Lkotlinx/coroutines/internal/LockFreeLinkedListNode$PrepareOp;)Ljava/lang/Object; } -public abstract interface class kotlinx/coroutines/selects/SelectClause0 { - public abstract fun registerSelectClause0 (Lkotlinx/coroutines/selects/SelectInstance;Lkotlin/jvm/functions/Function1;)V +public abstract interface class kotlinx/coroutines/selects/SelectClause { + public abstract fun getClauseObject ()Ljava/lang/Object; + public abstract fun getOnCancellationConstructor ()Lkotlin/jvm/functions/Function3; + public abstract fun getProcessResFunc ()Lkotlin/jvm/functions/Function3; + public abstract fun getRegFunc ()Lkotlin/jvm/functions/Function3; +} + +public abstract interface class kotlinx/coroutines/selects/SelectClause0 : kotlinx/coroutines/selects/SelectClause { +} + +public abstract interface class kotlinx/coroutines/selects/SelectClause1 : kotlinx/coroutines/selects/SelectClause { } -public abstract interface class kotlinx/coroutines/selects/SelectClause1 { - public abstract fun registerSelectClause1 (Lkotlinx/coroutines/selects/SelectInstance;Lkotlin/jvm/functions/Function2;)V +public abstract interface class kotlinx/coroutines/selects/SelectClause2 : kotlinx/coroutines/selects/SelectClause { } -public abstract interface class kotlinx/coroutines/selects/SelectClause2 { - public abstract fun registerSelectClause2 (Lkotlinx/coroutines/selects/SelectInstance;Ljava/lang/Object;Lkotlin/jvm/functions/Function2;)V +public class kotlinx/coroutines/selects/SelectImplementation : kotlinx/coroutines/selects/SelectBuilder, kotlinx/coroutines/selects/SelectInstanceInternal { + public fun <init> (Lkotlin/coroutines/CoroutineContext;)V + public fun disposeOnCompletion (Lkotlinx/coroutines/DisposableHandle;)V + public fun doSelect (Lkotlin/coroutines/Continuation;)Ljava/lang/Object; + public fun getContext ()Lkotlin/coroutines/CoroutineContext; + public synthetic fun invoke (Ljava/lang/Object;)Ljava/lang/Object; + public fun invoke (Ljava/lang/Throwable;)V + public fun invoke (Lkotlinx/coroutines/selects/SelectClause0;Lkotlin/jvm/functions/Function1;)V + public fun invoke (Lkotlinx/coroutines/selects/SelectClause1;Lkotlin/jvm/functions/Function2;)V + public fun invoke (Lkotlinx/coroutines/selects/SelectClause2;Ljava/lang/Object;Lkotlin/jvm/functions/Function2;)V + public fun invoke (Lkotlinx/coroutines/selects/SelectClause2;Lkotlin/jvm/functions/Function2;)V + public fun invokeOnCancellation (Lkotlinx/coroutines/internal/Segment;I)V + public fun onTimeout (JLkotlin/jvm/functions/Function1;)V + public fun selectInRegistrationPhase (Ljava/lang/Object;)V + public fun trySelect (Ljava/lang/Object;Ljava/lang/Object;)Z + public final fun trySelectDetailed (Ljava/lang/Object;Ljava/lang/Object;)Lkotlinx/coroutines/selects/TrySelectDetailedResult; } public abstract interface class kotlinx/coroutines/selects/SelectInstance { - public abstract fun disposeOnSelect (Lkotlinx/coroutines/DisposableHandle;)V - public abstract fun getCompletion ()Lkotlin/coroutines/Continuation; - public abstract fun isSelected ()Z - public abstract fun performAtomicTrySelect (Lkotlinx/coroutines/internal/AtomicDesc;)Ljava/lang/Object; - public abstract fun resumeSelectWithException (Ljava/lang/Throwable;)V - public abstract fun trySelect ()Z - public abstract fun trySelectOther (Lkotlinx/coroutines/internal/LockFreeLinkedListNode$PrepareOp;)Ljava/lang/Object; + public abstract fun disposeOnCompletion (Lkotlinx/coroutines/DisposableHandle;)V + public abstract fun getContext ()Lkotlin/coroutines/CoroutineContext; + public abstract fun selectInRegistrationPhase (Ljava/lang/Object;)V + public abstract fun trySelect (Ljava/lang/Object;Ljava/lang/Object;)Z } public final class kotlinx/coroutines/selects/SelectKt { - public static final fun onTimeout-8Mi8wO0 (Lkotlinx/coroutines/selects/SelectBuilder;JLkotlin/jvm/functions/Function1;)V public static final fun select (Lkotlin/jvm/functions/Function1;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; } +public final class kotlinx/coroutines/selects/SelectOldKt { + public static final fun selectOld (Lkotlin/jvm/functions/Function1;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; + public static final fun selectUnbiasedOld (Lkotlin/jvm/functions/Function1;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; +} + public final class kotlinx/coroutines/selects/SelectUnbiasedKt { public static final fun selectUnbiased (Lkotlin/jvm/functions/Function1;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; } -public final class kotlinx/coroutines/selects/UnbiasedSelectBuilderImpl : kotlinx/coroutines/selects/SelectBuilder { +public final class kotlinx/coroutines/selects/UnbiasedSelectBuilderImpl : kotlinx/coroutines/selects/UnbiasedSelectImplementation { public fun <init> (Lkotlin/coroutines/Continuation;)V - public final fun getClauses ()Ljava/util/ArrayList; - public final fun getInstance ()Lkotlinx/coroutines/selects/SelectBuilderImpl; public final fun handleBuilderException (Ljava/lang/Throwable;)V public final fun initSelectResult ()Ljava/lang/Object; +} + +public class kotlinx/coroutines/selects/UnbiasedSelectImplementation : kotlinx/coroutines/selects/SelectImplementation { + public fun <init> (Lkotlin/coroutines/CoroutineContext;)V + public fun doSelect (Lkotlin/coroutines/Continuation;)Ljava/lang/Object; public fun invoke (Lkotlinx/coroutines/selects/SelectClause0;Lkotlin/jvm/functions/Function1;)V public fun invoke (Lkotlinx/coroutines/selects/SelectClause1;Lkotlin/jvm/functions/Function2;)V public fun invoke (Lkotlinx/coroutines/selects/SelectClause2;Ljava/lang/Object;Lkotlin/jvm/functions/Function2;)V - public fun invoke (Lkotlinx/coroutines/selects/SelectClause2;Lkotlin/jvm/functions/Function2;)V - public fun onTimeout (JLkotlin/jvm/functions/Function1;)V } public final class kotlinx/coroutines/selects/WhileSelectKt { public static final fun whileSelect (Lkotlin/jvm/functions/Function1;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; } +public final class kotlinx/coroutines/stream/StreamKt { + public static final fun consumeAsFlow (Ljava/util/stream/Stream;)Lkotlinx/coroutines/flow/Flow; +} + public abstract interface class kotlinx/coroutines/sync/Mutex { public abstract fun getOnLock ()Lkotlinx/coroutines/selects/SelectClause2; public abstract fun holdsLock (Ljava/lang/Object;)Z @@ -1309,3 +1419,12 @@ public final class kotlinx/coroutines/sync/SemaphoreKt { public static final fun withPermit (Lkotlinx/coroutines/sync/Semaphore;Lkotlin/jvm/functions/Function0;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; } +public final class kotlinx/coroutines/time/TimeKt { + public static final fun debounce (Lkotlinx/coroutines/flow/Flow;Ljava/time/Duration;)Lkotlinx/coroutines/flow/Flow; + public static final fun delay (Ljava/time/Duration;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; + public static final fun onTimeout (Lkotlinx/coroutines/selects/SelectBuilder;Ljava/time/Duration;Lkotlin/jvm/functions/Function1;)V + public static final fun sample (Lkotlinx/coroutines/flow/Flow;Ljava/time/Duration;)Lkotlinx/coroutines/flow/Flow; + public static final fun withTimeout (Ljava/time/Duration;Lkotlin/jvm/functions/Function2;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; + public static final fun withTimeoutOrNull (Ljava/time/Duration;Lkotlin/jvm/functions/Function2;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; +} + diff --git a/kotlinx-coroutines-core/build.gradle b/kotlinx-coroutines-core/build.gradle index 9791b445..001df3a0 100644 --- a/kotlinx-coroutines-core/build.gradle +++ b/kotlinx-coroutines-core/build.gradle @@ -4,6 +4,10 @@ apply plugin: 'org.jetbrains.kotlin.multiplatform' apply plugin: 'org.jetbrains.dokka' + +// apply plugin to use autocomplete for Kover DSL +apply plugin: 'org.jetbrains.kotlinx.kover' + apply from: rootProject.file("gradle/compile-jvm-multiplatform.gradle") apply from: rootProject.file("gradle/compile-common.gradle") @@ -19,23 +23,50 @@ apply from: rootProject.file('gradle/publish.gradle') /* ========================================================================== Configure source sets structure for kotlinx-coroutines-core: - TARGETS SOURCE SETS - ------- ---------------------------------------------- + TARGETS SOURCE SETS + ------- ---------------------------------------------- js -----------------------------------------------------+ | V - jvm -------------------------------> concurrent ---> common - ^ - ios \ | - macos | ---> nativeDarwin ---> native --+ + jvmCore\ --------> jvm ---------> concurrent -------> common + jdk8 / ^ + | + ios \ | + macos | ---> nativeDarwin ---> native ---+ tvos | ^ watchos / | | linux \ ---> nativeOther -------+ mingw / - ========================================================================== */ + +Explanation of JVM source sets structure: + +The overall structure is just a hack to support the scenario we are interested in: + +* We would like to have two source-sets "core" and "jdk8" +* "jdk8" is allowed to use API from Java 8 and from "core" +* "core" is prohibited to use any API from "jdk8" +* It is okay to have tests in a single test source-set +* And we want to publish a **single** artifact kotlinx-coroutines-core.jar that contains classes from both source-sets +* Current limitation: only classes from "core" are checked with animal-sniffer + +For that, we have following compilations: +* jvmMain compilation: [jvmCoreMain, jdk8Main] +* jvmCore compilation: [commonMain] +* jdk8 compilation: [commonMain, jvmCoreMain] + +Theoretically, "jvmCore" could've been "jvmMain", it is not for technical reasons, +here is the explanation from Seb: + +""" +The jvmCore is theoretically not necessary. All code for jdk6 compatibility can be in jvmMain and jdk8 dependent code can be in jdk8Main. +Effectively there is no reason for ever putting code into jvmCoreMain. +However, when creating a new compilation, we have to take care of creating a defaultSourceSet. Without creating the jvmCoreMain source set, + the creation of the compilation fails. That is the only reason for this source set. +""" + ========================================================================== */ project.ext.sourceSetSuffixes = ["Main", "Test"] @@ -56,7 +87,8 @@ void defineSourceSet(newName, dependsOn, includedInPred) { } static boolean isNativeDarwin(String name) { return ["ios", "macos", "tvos", "watchos"].any { name.startsWith(it) } } -static boolean isNativeOther(String name) { return ["linux", "mingw"].any { name.startsWith(it) } } + +static boolean isNativeOther(String name) { return ["linux", "mingw", "androidNative"].any { name.startsWith(it) } } defineSourceSet("concurrent", ["common"]) { it in ["jvm", "native"] } @@ -67,49 +99,25 @@ if (rootProject.ext.native_targets_enabled) { /* ========================================================================== */ + /* * All platform plugins and configuration magic happens here instead of build.gradle * because JMV-only projects depend on core, thus core should always be initialized before configuration. */ kotlin { - sourceSets.forEach { - SourceSetsKt.configureMultiplatform(it) - } - /* - * Configure four test runs: - * 1) Old memory model, Main thread - * 2) New memory model, Main thread - * 3) Old memory model, BG thread - * 4) New memory model, BG thread (required for Dispatchers.Main tests on Darwin) + * Configure two test runs: + * 1) New memory model, Main thread + * 2) New memory model, BG thread (required for Dispatchers.Main tests on Darwin) * * All new MM targets are build with optimize = true to have stress tests properly run. */ targets.withType(org.jetbrains.kotlin.gradle.plugin.mpp.KotlinNativeTargetWithTests.class).configureEach { - binaries { + binaries.getTest("DEBUG").with { + optimized = true // Test for memory leaks using a special entry point that does not exit but returns from main - binaries.getTest("DEBUG").freeCompilerArgs += ["-e", "kotlinx.coroutines.mainNoExit"] - } - - binaries.test("newMM", [DEBUG]) { - def thisTest = it freeCompilerArgs += ["-e", "kotlinx.coroutines.mainNoExit"] - optimized = true binaryOptions["memoryModel"] = "experimental" - testRuns.create("newMM") { - setExecutionSourceFrom(thisTest) - // A hack to get different suffixes in the aggregated report. - executionTask.configure { targetName = "$targetName new MM" } - } - } - - binaries.test("worker", [DEBUG]) { - def thisTest = it - freeCompilerArgs += ["-e", "kotlinx.coroutines.mainBackground"] - testRuns.create("worker") { - setExecutionSourceFrom(thisTest) - executionTask.configure { targetName = "$targetName worker" } - } } binaries.test("workerWithNewMM", [DEBUG]) { @@ -124,13 +132,32 @@ kotlin { } } + def jvmMain = sourceSets.jvmMain + def jvmCoreMain = sourceSets.create('jvmCoreMain') + def jdk8Main = sourceSets.create('jdk8Main') + jvmCoreMain.dependsOn(jvmMain) + jdk8Main.dependsOn(jvmMain) + + sourceSets.forEach { + SourceSetsKt.configureMultiplatform(it) + } + jvm { + def main = compilations.main + main.source(jvmCoreMain) + main.source(jdk8Main) + + /* Create compilation for jvmCore to prove that jvmMain does not rely on jdk8 */ + compilations.create('CoreMain') { + /* jvmCore is automatically matched as 'defaultSourceSet' for the compilation, due to its name */ + tasks.getByName('check').dependsOn(compileKotlinTaskProvider) + } + // For animal sniffer withJava() } } - configurations { configureKotlinJvmPlatform(kotlinCompilerPluginClasspath) } @@ -150,11 +177,11 @@ def configureNativeSourceSetPreset(name, preset) { def implementationConfiguration = configurations[hostMainCompilation.defaultSourceSet.implementationMetadataConfigurationName] // Now find the libraries: Finds platform libs & stdlib, but platform declarations are still not resolved due to IDE bugs def hostNativePlatformLibs = files( - provider { - implementationConfiguration.findAll { - it.path.endsWith(".klib") || it.absolutePath.contains("klib${File.separator}platform") || it.absolutePath.contains("stdlib") + provider { + implementationConfiguration.findAll { + it.path.endsWith(".klib") || it.absolutePath.contains("klib${File.separator}platform") || it.absolutePath.contains("stdlib") + } } - } ) // Add all those dependencies for (suffix in sourceSetSuffixes) { @@ -188,6 +215,16 @@ kotlin.sourceSets { api "org.jetbrains.kotlinx:lincheck:$lincheck_version" api "org.jetbrains.kotlinx:kotlinx-knit-test:$knit_version" implementation project(":android-unit-tests") + implementation "org.openjdk.jol:jol-core:0.16" + } +} + +kotlin.sourceSets.configureEach { + // Do not apply 'ExperimentalForeignApi' where we have allWarningsAsErrors set + if (it.name in ["jvmMain", "jsMain", "concurrentMain", "commonMain"]) return + languageSettings { + optIn('kotlinx.cinterop.ExperimentalForeignApi') + optIn('kotlin.experimental.ExperimentalNativeApi') } } @@ -204,7 +241,7 @@ jvmTest { } // 'stress' is required to be able to run all subpackage tests like ":jvmTests --tests "*channels*" -Pstress=true" if (!Idea.active && rootProject.properties['stress'] == null) { - exclude '**/*LincheckTest.*' + exclude '**/*LincheckTest*' exclude '**/*StressTest.*' } if (Idea.active) { @@ -241,38 +278,53 @@ task jvmStressTest(type: Test, dependsOn: compileTestKotlinJvm) { enableAssertions = true testLogging.showStandardStreams = true systemProperty 'kotlinx.coroutines.scheduler.keep.alive.sec', '100000' // any unpark problem hangs test - systemProperty 'kotlinx.coroutines.semaphore.segmentSize', '2' + // Adjust internal algorithmic parameters to increase the testing quality instead of performance. + systemProperty 'kotlinx.coroutines.semaphore.segmentSize', '1' systemProperty 'kotlinx.coroutines.semaphore.maxSpinCycles', '10' + systemProperty 'kotlinx.coroutines.bufferedChannel.segmentSize', '2' + systemProperty 'kotlinx.coroutines.bufferedChannel.expandBufferCompletionWaitIterations', '1' } task jvmLincheckTest(type: Test, dependsOn: compileTestKotlinJvm) { classpath = files { jvmTest.classpath } testClassesDirs = files { jvmTest.testClassesDirs } - include '**/*LincheckTest.*' + include '**/*LincheckTest*' enableAssertions = true testLogging.showStandardStreams = true configureJvmForLincheck(jvmLincheckTest) } -static void configureJvmForLincheck(task) { +// Additional Lincheck tests with `segmentSize = 2`. +// Some bugs cannot be revealed when storing one request per segment, +// and some are hard to detect when storing multiple requests. +task jvmLincheckTestAdditional(type: Test, dependsOn: compileTestKotlinJvm) { + classpath = files { jvmTest.classpath } + testClassesDirs = files { jvmTest.testClassesDirs } + include '**/RendezvousChannelLincheckTest*' + include '**/Buffered1ChannelLincheckTest*' + include '**/Semaphore*LincheckTest*' + enableAssertions = true + testLogging.showStandardStreams = true + configureJvmForLincheck(jvmLincheckTestAdditional, true) +} + +static void configureJvmForLincheck(task, additional = false) { task.minHeapSize = '1g' task.maxHeapSize = '4g' // we may need more space for building an interleaving tree in the model checking mode task.jvmArgs = ['--add-opens', 'java.base/jdk.internal.misc=ALL-UNNAMED', // required for transformation '--add-exports', 'java.base/jdk.internal.util=ALL-UNNAMED'] // in the model checking mode - task.systemProperty 'kotlinx.coroutines.semaphore.segmentSize', '2' + // Adjust internal algorithmic parameters to increase the testing quality instead of performance. + var segmentSize = additional ? '2' : '1' + task.systemProperty 'kotlinx.coroutines.semaphore.segmentSize', segmentSize task.systemProperty 'kotlinx.coroutines.semaphore.maxSpinCycles', '1' // better for the model checking mode + task.systemProperty 'kotlinx.coroutines.bufferedChannel.segmentSize', segmentSize + task.systemProperty 'kotlinx.coroutines.bufferedChannel.expandBufferCompletionWaitIterations', '1' } // Always check additional test sets -task moreTest(dependsOn: [jvmStressTest, jvmLincheckTest]) +task moreTest(dependsOn: [jvmStressTest, jvmLincheckTest, jvmLincheckTestAdditional]) check.dependsOn moreTest -tasks.jvmLincheckTest { - kover { - enabled = false // Always disabled, lincheck doesn't really support coverage - } -} - def commonKoverExcludes = ["kotlinx.coroutines.debug.*", // Tested by debug module "kotlinx.coroutines.channels.ChannelsKt__DeprecatedKt.*", // Deprecated @@ -280,17 +332,34 @@ def commonKoverExcludes = "kotlinx.coroutines.scheduling.ExperimentalCoroutineDispatcher" // Deprecated ] -tasks.koverHtmlReport { - excludes = commonKoverExcludes +kover { + excludeTests { + // Always disabled, lincheck doesn't really support coverage + tasks("jvmLincheckTest") + } + + excludeInstrumentation { + // lincheck has NPE error on `ManagedStrategyStateHolder` class + classes("org.jetbrains.kotlinx.lincheck.*") + } } -tasks.koverVerify { - excludes = commonKoverExcludes +koverReport { + filters { + excludes { + classes( + "kotlinx.coroutines.debug.*", // Tested by debug module + "kotlinx.coroutines.channels.ChannelsKt__DeprecatedKt.*", // Deprecated + "kotlinx.coroutines.scheduling.LimitingDispatcher", // Deprecated + "kotlinx.coroutines.scheduling.ExperimentalCoroutineDispatcher" // Deprecated + ) + } + } } task testsJar(type: Jar, dependsOn: jvmTestClasses) { classifier = 'tests' - from compileTestKotlinJvm.destinationDir + from(compileTestKotlinJvm.destinationDirectory) } artifacts { diff --git a/kotlinx-coroutines-core/common/src/Builders.common.kt b/kotlinx-coroutines-core/common/src/Builders.common.kt index 3dea68cf..bcb2dd81 100644 --- a/kotlinx-coroutines-core/common/src/Builders.common.kt +++ b/kotlinx-coroutines-core/common/src/Builders.common.kt @@ -96,12 +96,10 @@ public fun <T> CoroutineScope.async( private open class DeferredCoroutine<T>( parentContext: CoroutineContext, active: Boolean -) : AbstractCoroutine<T>(parentContext, true, active = active), Deferred<T>, SelectClause1<T> { +) : AbstractCoroutine<T>(parentContext, true, active = active), Deferred<T> { override fun getCompleted(): T = getCompletedInternal() as T override suspend fun await(): T = awaitInternal() as T - override val onAwait: SelectClause1<T> get() = this - override fun <R> registerSelectClause1(select: SelectInstance<R>, block: suspend (T) -> R) = - registerSelectClause1Internal(select, block) + override val onAwait: SelectClause1<T> get() = onAwaitInternal as SelectClause1<T> } private class LazyDeferredCoroutine<T>( @@ -165,7 +163,7 @@ public suspend fun <T> withContext( if (newContext[ContinuationInterceptor] == oldContext[ContinuationInterceptor]) { val coroutine = UndispatchedCoroutine(newContext, uCont) // There are changes in the context, so this thread needs to be updated - withCoroutineContext(newContext, null) { + withCoroutineContext(coroutine.context, null) { return@sc coroutine.startUndispatchedOrReturn(coroutine, block) } } @@ -220,13 +218,16 @@ private const val SUSPENDED = 1 private const val RESUMED = 2 // Used by withContext when context dispatcher changes -internal class DispatchedCoroutine<in T>( +@PublishedApi +internal class DispatchedCoroutine<in T> internal constructor( context: CoroutineContext, uCont: Continuation<T> ) : ScopeCoroutine<T>(context, uCont) { // this is copy-and-paste of a decision state machine inside AbstractionContinuation // todo: we may some-how abstract it via inline class - private val _decision = atomic(UNDECIDED) + // Used by the IDEA debugger via reflection and must be kept binary-compatible, see KTIJ-24102 + @JvmField + public val _decision = atomic(UNDECIDED) private fun trySuspend(): Boolean { _decision.loop { decision -> @@ -260,7 +261,7 @@ internal class DispatchedCoroutine<in T>( uCont.intercepted().resumeCancellableWith(recoverResult(state, uCont)) } - fun getResult(): Any? { + internal fun getResult(): Any? { if (trySuspend()) return COROUTINE_SUSPENDED // otherwise, onCompletionInternal was already invoked & invoked tryResume, and the result is in the state val state = this.state.unboxState() diff --git a/kotlinx-coroutines-core/common/src/CancellableContinuation.kt b/kotlinx-coroutines-core/common/src/CancellableContinuation.kt index 2c2f1b8f..e3237e57 100644 --- a/kotlinx-coroutines-core/common/src/CancellableContinuation.kt +++ b/kotlinx-coroutines-core/common/src/CancellableContinuation.kt @@ -328,15 +328,22 @@ public suspend inline fun <T> suspendCancellableCoroutine( * [CancellableContinuationImpl] is reused. */ internal suspend inline fun <T> suspendCancellableCoroutineReusable( - crossinline block: (CancellableContinuation<T>) -> Unit + crossinline block: (CancellableContinuationImpl<T>) -> Unit ): T = suspendCoroutineUninterceptedOrReturn { uCont -> val cancellable = getOrCreateCancellableContinuation(uCont.intercepted()) - block(cancellable) + try { + block(cancellable) + } catch (e: Throwable) { + // Here we catch any unexpected exception from user-supplied block (e.g. invariant violation) + // and release claimed continuation in order to leave it in a reasonable state (see #3613) + cancellable.releaseClaimedReusableContinuation() + throw e + } cancellable.getResult() } internal fun <T> getOrCreateCancellableContinuation(delegate: Continuation<T>): CancellableContinuationImpl<T> { - // If used outside of our dispatcher + // If used outside our dispatcher if (delegate !is DispatchedContinuation<T>) { return CancellableContinuationImpl(delegate, MODE_CANCELLABLE) } @@ -359,13 +366,6 @@ internal fun <T> getOrCreateCancellableContinuation(delegate: Continuation<T>): } /** - * Removes the specified [node] on cancellation. This function assumes that this node is already - * removed on successful resume and does not try to remove it if the continuation is cancelled during dispatch. - */ -internal fun CancellableContinuation<*>.removeOnCancellation(node: LockFreeLinkedListNode) = - invokeOnCancellation(handler = RemoveOnCancel(node).asHandler) - -/** * Disposes the specified [handle] when this continuation is cancelled. * * This is a shortcut for the following code with slightly more efficient implementation (one fewer object created): @@ -379,13 +379,6 @@ internal fun CancellableContinuation<*>.removeOnCancellation(node: LockFreeLinke public fun CancellableContinuation<*>.disposeOnCancellation(handle: DisposableHandle): Unit = invokeOnCancellation(handler = DisposeOnCancel(handle).asHandler) -// --------------- implementation details --------------- - -private class RemoveOnCancel(private val node: LockFreeLinkedListNode) : BeforeResumeCancelHandler() { - override fun invoke(cause: Throwable?) { node.remove() } - override fun toString() = "RemoveOnCancel[$node]" -} - private class DisposeOnCancel(private val handle: DisposableHandle) : CancelHandler() { override fun invoke(cause: Throwable?) = handle.dispose() override fun toString(): String = "DisposeOnCancel[$handle]" diff --git a/kotlinx-coroutines-core/common/src/CancellableContinuationImpl.kt b/kotlinx-coroutines-core/common/src/CancellableContinuationImpl.kt index 1a0169b6..ed2d9f20 100644 --- a/kotlinx-coroutines-core/common/src/CancellableContinuationImpl.kt +++ b/kotlinx-coroutines-core/common/src/CancellableContinuationImpl.kt @@ -9,14 +9,21 @@ import kotlinx.coroutines.internal.* import kotlin.coroutines.* import kotlin.coroutines.intrinsics.* import kotlin.jvm.* -import kotlin.native.concurrent.* private const val UNDECIDED = 0 private const val SUSPENDED = 1 private const val RESUMED = 2 +private const val DECISION_SHIFT = 29 +private const val INDEX_MASK = (1 shl DECISION_SHIFT) - 1 +private const val NO_INDEX = INDEX_MASK + +private inline val Int.decision get() = this shr DECISION_SHIFT +private inline val Int.index get() = this and INDEX_MASK +@Suppress("NOTHING_TO_INLINE") +private inline fun decisionAndIndex(decision: Int, index: Int) = (decision shl DECISION_SHIFT) + index + @JvmField -@SharedImmutable internal val RESUME_TOKEN = Symbol("RESUME_TOKEN") /** @@ -26,7 +33,7 @@ internal val RESUME_TOKEN = Symbol("RESUME_TOKEN") internal open class CancellableContinuationImpl<in T>( final override val delegate: Continuation<T>, resumeMode: Int -) : DispatchedTask<T>(resumeMode), CancellableContinuation<T>, CoroutineStackFrame { +) : DispatchedTask<T>(resumeMode), CancellableContinuation<T>, CoroutineStackFrame, Waiter { init { assert { resumeMode != MODE_UNINITIALIZED } // invalid mode for CancellableContinuationImpl } @@ -45,7 +52,7 @@ internal open class CancellableContinuationImpl<in T>( * less dependencies. */ - /* decision state machine + /** decision state machine +-----------+ trySuspend +-----------+ | UNDECIDED | -------------> | SUSPENDED | @@ -57,9 +64,12 @@ internal open class CancellableContinuationImpl<in T>( | RESUMED | +-----------+ - Note: both tryResume and trySuspend can be invoked at most once, first invocation wins + Note: both tryResume and trySuspend can be invoked at most once, first invocation wins. + If the cancellation handler is specified via a [Segment] instance and the index in it + (so [Segment.onCancellation] should be called), the [_decisionAndIndex] field may store + this index additionally to the "decision" value. */ - private val _decision = atomic(UNDECIDED) + private val _decisionAndIndex = atomic(decisionAndIndex(UNDECIDED, NO_INDEX)) /* === Internal states === @@ -72,7 +82,28 @@ internal open class CancellableContinuationImpl<in T>( */ private val _state = atomic<Any?>(Active) - private var parentHandle: DisposableHandle? = null + /* + * This field has a concurrent rendezvous in the following scenario: + * + * - installParentHandle publishes this instance on T1 + * + * T1 writes: + * * handle = installed; right after the installation + * * Shortly after: if (isComplete) handle = NonDisposableHandle + * + * Any other T writes if the parent job is cancelled in detachChild: + * * handle = NonDisposableHandle + * + * We want to preserve a strict invariant on parentHandle transition, allowing only three of them: + * null -> anyHandle + * anyHandle -> NonDisposableHandle + * null -> NonDisposableHandle + * + * With a guarantee that after disposal the only state handle may end up in is NonDisposableHandle + */ + private val _parentHandle = atomic<DisposableHandle?>(null) + private val parentHandle: DisposableHandle? + get() = _parentHandle.value internal val state: Any? get() = _state.value @@ -103,7 +134,7 @@ internal open class CancellableContinuationImpl<in T>( if (isCompleted) { // Can be invoked concurrently in 'parentCancelled', no problems here handle.dispose() - parentHandle = NonDisposableHandle + _parentHandle.value = NonDisposableHandle } } @@ -124,7 +155,7 @@ internal open class CancellableContinuationImpl<in T>( detachChild() return false } - _decision.value = UNDECIDED + _decisionAndIndex.value = decisionAndIndex(UNDECIDED, NO_INDEX) _state.value = Active return true } @@ -174,10 +205,13 @@ internal open class CancellableContinuationImpl<in T>( _state.loop { state -> if (state !is NotCompleted) return false // false if already complete or cancelling // Active -- update to final state - val update = CancelledContinuation(this, cause, handled = state is CancelHandler) + val update = CancelledContinuation(this, cause, handled = state is CancelHandler || state is Segment<*>) if (!_state.compareAndSet(state, update)) return@loop // retry on cas failure // Invoke cancel handler if it was present - (state as? CancelHandler)?.let { callCancelHandler(it, cause) } + when (state) { + is CancelHandler -> callCancelHandler(state, cause) + is Segment<*> -> callSegmentOnCancellation(state, cause) + } // Complete state update detachChildIfNonResuable() dispatchResume(resumeMode) // no need for additional cancellation checks @@ -214,6 +248,12 @@ internal open class CancellableContinuationImpl<in T>( fun callCancelHandler(handler: CancelHandler, cause: Throwable?) = callCancelHandlerSafely { handler.invoke(cause) } + private fun callSegmentOnCancellation(segment: Segment<*>, cause: Throwable?) { + val index = _decisionAndIndex.value.index + check(index != NO_INDEX) { "The index for Segment.onCancellation(..) is broken" } + callCancelHandlerSafely { segment.onCancellation(index, cause, context) } + } + fun callOnCancellation(onCancellation: (cause: Throwable) -> Unit, cause: Throwable) { try { onCancellation.invoke(cause) @@ -233,9 +273,9 @@ internal open class CancellableContinuationImpl<in T>( parent.getCancellationException() private fun trySuspend(): Boolean { - _decision.loop { decision -> - when (decision) { - UNDECIDED -> if (this._decision.compareAndSet(UNDECIDED, SUSPENDED)) return true + _decisionAndIndex.loop { cur -> + when (cur.decision) { + UNDECIDED -> if (this._decisionAndIndex.compareAndSet(cur, decisionAndIndex(SUSPENDED, cur.index))) return true RESUMED -> return false else -> error("Already suspended") } @@ -243,9 +283,9 @@ internal open class CancellableContinuationImpl<in T>( } private fun tryResume(): Boolean { - _decision.loop { decision -> - when (decision) { - UNDECIDED -> if (this._decision.compareAndSet(UNDECIDED, RESUMED)) return true + _decisionAndIndex.loop { cur -> + when (cur.decision) { + UNDECIDED -> if (this._decisionAndIndex.compareAndSet(cur, decisionAndIndex(RESUMED, cur.index))) return true SUSPENDED -> return false else -> error("Already resumed") } @@ -255,7 +295,7 @@ internal open class CancellableContinuationImpl<in T>( @PublishedApi internal fun getResult(): Any? { val isReusable = isReusable() - // trySuspend may fail either if 'block' has resumed/cancelled a continuation + // trySuspend may fail either if 'block' has resumed/cancelled a continuation, // or we got async cancellation from parent. if (trySuspend()) { /* @@ -309,7 +349,7 @@ internal open class CancellableContinuationImpl<in T>( onCancelling = true, handler = ChildContinuation(this).asHandler ) - parentHandle = handle + _parentHandle.compareAndSet(null, handle) return handle } @@ -317,8 +357,8 @@ internal open class CancellableContinuationImpl<in T>( * Tries to release reusable continuation. It can fail is there was an asynchronous cancellation, * in which case it detaches from the parent and cancels this continuation. */ - private fun releaseClaimedReusableContinuation() { - // Cannot be casted if e.g. invoked from `installParentHandleReusable` for context without dispatchers, but with Job in it + internal fun releaseClaimedReusableContinuation() { + // Cannot be cast if e.g. invoked from `installParentHandleReusable` for context without dispatchers, but with Job in it val cancellationCause = (delegate as? DispatchedContinuation<*>)?.tryReleaseClaimedContinuation(this) ?: return detachChild() cancel(cancellationCause) @@ -330,14 +370,43 @@ internal open class CancellableContinuationImpl<in T>( override fun resume(value: T, onCancellation: ((cause: Throwable) -> Unit)?) = resumeImpl(value, resumeMode, onCancellation) + /** + * An optimized version for the code below that does not allocate + * a cancellation handler object and efficiently stores the specified + * [segment] and [index] in this [CancellableContinuationImpl]. + * + * The only difference is that `segment.onCancellation(..)` is never + * called if this continuation is already completed; + * + * ``` + * invokeOnCancellation { cause -> + * segment.onCancellation(index, cause) + * } + * ``` + */ + override fun invokeOnCancellation(segment: Segment<*>, index: Int) { + _decisionAndIndex.update { + check(it.index == NO_INDEX) { + "invokeOnCancellation should be called at most once" + } + decisionAndIndex(it.decision, index) + } + invokeOnCancellationImpl(segment) + } + public override fun invokeOnCancellation(handler: CompletionHandler) { val cancelHandler = makeCancelHandler(handler) + invokeOnCancellationImpl(cancelHandler) + } + + private fun invokeOnCancellationImpl(handler: Any) { + assert { handler is CancelHandler || handler is Segment<*> } _state.loop { state -> when (state) { is Active -> { - if (_state.compareAndSet(state, cancelHandler)) return // quit on cas success + if (_state.compareAndSet(state, handler)) return // quit on cas success } - is CancelHandler -> multipleHandlersError(handler, state) + is CancelHandler, is Segment<*> -> multipleHandlersError(handler, state) is CompletedExceptionally -> { /* * Continuation was already cancelled or completed exceptionally. @@ -351,7 +420,13 @@ internal open class CancellableContinuationImpl<in T>( * because we play type tricks on Kotlin/JS and handler is not necessarily a function there */ if (state is CancelledContinuation) { - callCancelHandler(handler, (state as? CompletedExceptionally)?.cause) + val cause: Throwable? = (state as? CompletedExceptionally)?.cause + if (handler is CancelHandler) { + callCancelHandler(handler, cause) + } else { + val segment = handler as Segment<*> + callSegmentOnCancellation(segment, cause) + } } return } @@ -360,31 +435,33 @@ internal open class CancellableContinuationImpl<in T>( * Continuation was already completed, and might already have cancel handler. */ if (state.cancelHandler != null) multipleHandlersError(handler, state) - // BeforeResumeCancelHandler does not need to be called on a completed continuation - if (cancelHandler is BeforeResumeCancelHandler) return + // Segment.invokeOnCancellation(..) does NOT need to be called on completed continuation. + if (handler is Segment<*>) return + handler as CancelHandler if (state.cancelled) { // Was already cancelled while being dispatched -- invoke the handler directly callCancelHandler(handler, state.cancelCause) return } - val update = state.copy(cancelHandler = cancelHandler) + val update = state.copy(cancelHandler = handler) if (_state.compareAndSet(state, update)) return // quit on cas success } else -> { /* * Continuation was already completed normally, but might get cancelled while being dispatched. - * Change its state to CompletedContinuation, unless we have BeforeResumeCancelHandler which + * Change its state to CompletedContinuation, unless we have Segment which * does not need to be called in this case. */ - if (cancelHandler is BeforeResumeCancelHandler) return - val update = CompletedContinuation(state, cancelHandler = cancelHandler) + if (handler is Segment<*>) return + handler as CancelHandler + val update = CompletedContinuation(state, cancelHandler = handler) if (_state.compareAndSet(state, update)) return // quit on cas success } } } } - private fun multipleHandlersError(handler: CompletionHandler, state: Any?) { + private fun multipleHandlersError(handler: Any, state: Any?) { error("It's prohibited to register multiple handlers, tried to register $handler, already has $state") } @@ -410,7 +487,7 @@ internal open class CancellableContinuationImpl<in T>( proposedUpdate } !resumeMode.isCancellableMode && idempotent == null -> proposedUpdate // cannot be cancelled in process, all is fine - onCancellation != null || (state is CancelHandler && state !is BeforeResumeCancelHandler) || idempotent != null -> + onCancellation != null || state is CancelHandler || idempotent != null -> // mark as CompletedContinuation if special cases are present: // Cancellation handlers that shall be called after resume or idempotent resume CompletedContinuation(proposedUpdate, state as? CancelHandler, onCancellation, idempotent) @@ -494,7 +571,7 @@ internal open class CancellableContinuationImpl<in T>( internal fun detachChild() { val handle = parentHandle ?: return handle.dispose() - parentHandle = NonDisposableHandle + _parentHandle.value = NonDisposableHandle } // Note: Always returns RESUME_TOKEN | null @@ -557,14 +634,6 @@ private object Active : NotCompleted { */ internal abstract class CancelHandler : CancelHandlerBase(), NotCompleted -/** - * Base class for all [CancellableContinuation.invokeOnCancellation] handlers that don't need to be invoked - * if continuation is cancelled after resumption, during dispatch, because the corresponding resources - * were already released before calling `resume`. This cancel handler is called only before `resume`. - * It avoids allocation of [CompletedContinuation] instance during resume on JVM. - */ -internal abstract class BeforeResumeCancelHandler : CancelHandler() - // Wrapper for lambdas, for the performance sake CancelHandler can be subclassed directly private class InvokeOnCancel( // Clashes with InvokeOnCancellation private val handler: CompletionHandler diff --git a/kotlinx-coroutines-core/common/src/CloseableCoroutineDispatcher.kt b/kotlinx-coroutines-core/common/src/CloseableCoroutineDispatcher.kt index 9c670329..541b3082 100644 --- a/kotlinx-coroutines-core/common/src/CloseableCoroutineDispatcher.kt +++ b/kotlinx-coroutines-core/common/src/CloseableCoroutineDispatcher.kt @@ -19,8 +19,8 @@ public expect abstract class CloseableCoroutineDispatcher() : CoroutineDispatche /** * Initiate the closing sequence of the coroutine dispatcher. - * After a successful call to [close], no new tasks will - * be accepted to be [dispatched][dispatch], but the previously dispatched tasks will be run. + * After a successful call to [close], no new tasks will be accepted to be [dispatched][dispatch]. + * The previously-submitted tasks will still be run, but [close] is not guaranteed to wait for them to finish. * * Invocations of `close` are idempotent and thread-safe. */ diff --git a/kotlinx-coroutines-core/common/src/CompletableDeferred.kt b/kotlinx-coroutines-core/common/src/CompletableDeferred.kt index 5e76593d..293c5167 100644 --- a/kotlinx-coroutines-core/common/src/CompletableDeferred.kt +++ b/kotlinx-coroutines-core/common/src/CompletableDeferred.kt @@ -79,14 +79,12 @@ public fun <T> CompletableDeferred(value: T): CompletableDeferred<T> = Completab @Suppress("UNCHECKED_CAST") private class CompletableDeferredImpl<T>( parent: Job? -) : JobSupport(true), CompletableDeferred<T>, SelectClause1<T> { +) : JobSupport(true), CompletableDeferred<T> { init { initParentJob(parent) } override val onCancelComplete get() = true override fun getCompleted(): T = getCompletedInternal() as T override suspend fun await(): T = awaitInternal() as T - override val onAwait: SelectClause1<T> get() = this - override fun <R> registerSelectClause1(select: SelectInstance<R>, block: suspend (T) -> R) = - registerSelectClause1Internal(select, block) + override val onAwait: SelectClause1<T> get() = onAwaitInternal as SelectClause1<T> override fun complete(value: T): Boolean = makeCompleting(value) diff --git a/kotlinx-coroutines-core/common/src/CompletionState.kt b/kotlinx-coroutines-core/common/src/CompletionState.kt index b9042874..43330af4 100644 --- a/kotlinx-coroutines-core/common/src/CompletionState.kt +++ b/kotlinx-coroutines-core/common/src/CompletionState.kt @@ -40,7 +40,7 @@ internal data class CompletedWithCancellation( * or artificial [CancellationException] if no cause was provided */ internal open class CompletedExceptionally( - @JvmField public val cause: Throwable, + @JvmField val cause: Throwable, handled: Boolean = false ) { private val _handled = atomic(handled) diff --git a/kotlinx-coroutines-core/common/src/CoroutineExceptionHandler.kt b/kotlinx-coroutines-core/common/src/CoroutineExceptionHandler.kt index 49923a92..e641447b 100644 --- a/kotlinx-coroutines-core/common/src/CoroutineExceptionHandler.kt +++ b/kotlinx-coroutines-core/common/src/CoroutineExceptionHandler.kt @@ -4,10 +4,9 @@ package kotlinx.coroutines +import kotlinx.coroutines.internal.* import kotlin.coroutines.* -internal expect fun handleCoroutineExceptionImpl(context: CoroutineContext, exception: Throwable) - /** * Helper function for coroutine builder implementations to handle uncaught and unexpected exceptions in coroutines, * that could not be otherwise handled in a normal way through structured concurrency, saving them to a future, and @@ -26,11 +25,11 @@ public fun handleCoroutineException(context: CoroutineContext, exception: Throwa return } } catch (t: Throwable) { - handleCoroutineExceptionImpl(context, handlerException(exception, t)) + handleUncaughtCoroutineException(context, handlerException(exception, t)) return } // If a handler is not present in the context or an exception was thrown, fallback to the global handler - handleCoroutineExceptionImpl(context, exception) + handleUncaughtCoroutineException(context, exception) } internal fun handlerException(originalException: Throwable, thrownException: Throwable): Throwable { @@ -83,15 +82,16 @@ public inline fun CoroutineExceptionHandler(crossinline handler: (CoroutineConte * } * ``` * - * ### Implementation details + * ### Uncaught exceptions with no handler * - * By default, when no handler is installed, uncaught exception are handled in the following way: - * * If exception is [CancellationException] then it is ignored - * (because that is the supposed mechanism to cancel the running coroutine) - * * Otherwise: - * * if there is a [Job] in the context, then [Job.cancel] is invoked; - * * Otherwise, all instances of [CoroutineExceptionHandler] found via [ServiceLoader] - * * and current thread's [Thread.uncaughtExceptionHandler] are invoked. + * When no handler is installed, exception are handled in the following way: + * * If exception is [CancellationException], it is ignored, as these exceptions are used to cancel coroutines. + * * Otherwise, if there is a [Job] in the context, then [Job.cancel] is invoked. + * * Otherwise, as a last resort, the exception is processed in a platform-specific manner: + * - On JVM, all instances of [CoroutineExceptionHandler] found via [ServiceLoader], as well as + * the current thread's [Thread.uncaughtExceptionHandler], are invoked. + * - On Native, the whole application crashes with the exception. + * - On JS, the exception is logged via the Console API. * * [CoroutineExceptionHandler] can be invoked from an arbitrary thread. */ diff --git a/kotlinx-coroutines-core/common/src/Deferred.kt b/kotlinx-coroutines-core/common/src/Deferred.kt index 595700e2..2f106e9e 100644 --- a/kotlinx-coroutines-core/common/src/Deferred.kt +++ b/kotlinx-coroutines-core/common/src/Deferred.kt @@ -22,7 +22,7 @@ import kotlinx.coroutines.selects.* * Usually, a deferred value is created in _active_ state (it is created and started). * However, the [async][CoroutineScope.async] coroutine builder has an optional `start` parameter that creates a deferred value in _new_ state * when this parameter is set to [CoroutineStart.LAZY]. - * Such a deferred can be be made _active_ by invoking [start], [join], or [await]. + * Such a deferred can be made _active_ by invoking [start], [join], or [await]. * * A deferred value is a [Job]. A job in the * [coroutineContext](https://kotlinlang.org/api/latest/jvm/stdlib/kotlin.coroutines/coroutine-context.html) diff --git a/kotlinx-coroutines-core/common/src/Delay.kt b/kotlinx-coroutines-core/common/src/Delay.kt index 301ed2d3..ba06d977 100644 --- a/kotlinx-coroutines-core/common/src/Delay.kt +++ b/kotlinx-coroutines-core/common/src/Delay.kt @@ -57,6 +57,19 @@ public interface Delay { } /** + * Enhanced [Delay] interface that provides additional diagnostics for [withTimeout]. + * Is going to be removed once there is proper JVM-default support. + * Then we'll be able put this function into [Delay] without breaking binary compatibility. + */ +@InternalCoroutinesApi +internal interface DelayWithTimeoutDiagnostics : Delay { + /** + * Returns a string that explains that the timeout has occurred, and explains what can be done about it. + */ + fun timeoutMessage(timeout: Duration): String +} + +/** * Suspends until cancellation, in which case it will throw a [CancellationException]. * * This function returns [Nothing], so it can be used in any coroutine, @@ -94,6 +107,7 @@ public suspend fun awaitCancellation(): Nothing = suspendCancellableCoroutine {} /** * Delays coroutine for a given time without blocking a thread and resumes it after a specified time. + * If the given [timeMillis] is non-positive, this function returns immediately. * * This suspending function is cancellable. * If the [Job] of the current coroutine is cancelled or completed while this suspending function is waiting, this function @@ -120,6 +134,7 @@ public suspend fun delay(timeMillis: Long) { /** * Delays coroutine for a given [duration] without blocking a thread and resumes it after the specified time. + * If the given [duration] is non-positive, this function returns immediately. * * This suspending function is cancellable. * If the [Job] of the current coroutine is cancelled or completed while this suspending function is waiting, this function diff --git a/kotlinx-coroutines-core/common/src/Dispatchers.common.kt b/kotlinx-coroutines-core/common/src/Dispatchers.common.kt index 28e67a42..bb85d869 100644 --- a/kotlinx-coroutines-core/common/src/Dispatchers.common.kt +++ b/kotlinx-coroutines-core/common/src/Dispatchers.common.kt @@ -12,10 +12,10 @@ import kotlin.coroutines.* public expect object Dispatchers { /** * The default [CoroutineDispatcher] that is used by all standard builders like - * [launch][CoroutineScope.launch], [async][CoroutineScope.async], etc + * [launch][CoroutineScope.launch], [async][CoroutineScope.async], etc. * if neither a dispatcher nor any other [ContinuationInterceptor] is specified in their context. * - * It is backed by a shared pool of threads on JVM. By default, the maximum number of threads used + * It is backed by a shared pool of threads on JVM and Native. By default, the maximum number of threads used * by this dispatcher is equal to the number of CPU cores, but is at least two. */ public val Default: CoroutineDispatcher @@ -27,16 +27,17 @@ public expect object Dispatchers { * Access to this property may throw an [IllegalStateException] if no main dispatchers are present in the classpath. * * Depending on platform and classpath, it can be mapped to different dispatchers: - * - On JS and Native it is equivalent to the [Default] dispatcher. - * - On JVM it is either the Android main thread dispatcher, JavaFx or Swing EDT dispatcher. It is chosen by the + * - On JVM it is either the Android main thread dispatcher, JavaFx, or Swing EDT dispatcher. It is chosen by the * [`ServiceLoader`](https://docs.oracle.com/javase/8/docs/api/java/util/ServiceLoader.html). + * - On JS it is equivalent to the [Default] dispatcher with [immediate][MainCoroutineDispatcher.immediate] support. + * - On Native Darwin-based targets, it is a dispatcher backed by Darwin's main queue. + * - On other Native targets, it is a single-threaded dispatcher backed by a standalone worker. * * In order to work with the `Main` dispatcher, the following artifact should be added to the project runtime dependencies: * - `kotlinx-coroutines-android` — for Android Main thread dispatcher * - `kotlinx-coroutines-javafx` — for JavaFx Application thread dispatcher * - `kotlinx-coroutines-swing` — for Swing EDT dispatcher - * - * Implementation note: [MainCoroutineDispatcher.immediate] is not supported on the Native and JS platforms. + * - `kotlinx-coroutines-test` — for mocking the `Main` dispatcher in tests via `Dispatchers.setMain` */ public val Main: MainCoroutineDispatcher @@ -56,7 +57,7 @@ public expect object Dispatchers { * ``` * withContext(Dispatchers.Unconfined) { * println(1) - * withContext(Dispatchers.Unconfined) { // Nested unconfined + * launch(Dispatchers.Unconfined) { // Nested unconfined * println(2) * } * println(3) @@ -64,7 +65,7 @@ public expect object Dispatchers { * println("Done") * ``` * Can print both "1 2 3" and "1 3 2". This is an implementation detail that can be changed. - * However, it is guaranteed that "Done" will be printed only when both `withContext` calls are completed. + * However, it is guaranteed that "Done" will only be printed once the code in both `withContext` and `launch` completes. * * If you need your coroutine to be confined to a particular thread or a thread-pool after resumption, * but still want to execute it in the current call-frame until its first suspension, you can use diff --git a/kotlinx-coroutines-core/common/src/EventLoop.common.kt b/kotlinx-coroutines-core/common/src/EventLoop.common.kt index 12940c54..8d9eed21 100644 --- a/kotlinx-coroutines-core/common/src/EventLoop.common.kt +++ b/kotlinx-coroutines-core/common/src/EventLoop.common.kt @@ -8,7 +8,6 @@ import kotlinx.atomicfu.* import kotlinx.coroutines.internal.* import kotlin.coroutines.* import kotlin.jvm.* -import kotlin.native.concurrent.* /** * Extended by [CoroutineDispatcher] implementations that have event loop inside and can @@ -37,7 +36,7 @@ internal abstract class EventLoop : CoroutineDispatcher() { * Queue used by [Dispatchers.Unconfined] tasks. * These tasks are thread-local for performance and take precedence over the rest of the queue. */ - private var unconfinedQueue: ArrayQueue<DispatchedTask<*>>? = null + private var unconfinedQueue: ArrayDeque<DispatchedTask<*>>? = null /** * Processes next event in this event loop. @@ -50,7 +49,7 @@ internal abstract class EventLoop : CoroutineDispatcher() { * **NOTE**: Must be invoked only from the event loop's thread * (no check for performance reasons, may be added in the future). */ - public open fun processNextEvent(): Long { + open fun processNextEvent(): Long { if (!processUnconfinedEvent()) return Long.MAX_VALUE return 0 } @@ -60,10 +59,10 @@ internal abstract class EventLoop : CoroutineDispatcher() { protected open val nextTime: Long get() { val queue = unconfinedQueue ?: return Long.MAX_VALUE - return if (queue.isEmpty) Long.MAX_VALUE else 0L + return if (queue.isEmpty()) Long.MAX_VALUE else 0L } - public fun processUnconfinedEvent(): Boolean { + fun processUnconfinedEvent(): Boolean { val queue = unconfinedQueue ?: return false val task = queue.removeFirstOrNull() ?: return false task.run() @@ -75,27 +74,27 @@ internal abstract class EventLoop : CoroutineDispatcher() { * By default, event loop implementation is thread-local and should not processed in the context * (current thread's event loop should be processed instead). */ - public open fun shouldBeProcessedFromContext(): Boolean = false + open fun shouldBeProcessedFromContext(): Boolean = false /** * Dispatches task whose dispatcher returned `false` from [CoroutineDispatcher.isDispatchNeeded] * into the current event loop. */ - public fun dispatchUnconfined(task: DispatchedTask<*>) { + fun dispatchUnconfined(task: DispatchedTask<*>) { val queue = unconfinedQueue ?: - ArrayQueue<DispatchedTask<*>>().also { unconfinedQueue = it } + ArrayDeque<DispatchedTask<*>>().also { unconfinedQueue = it } queue.addLast(task) } - public val isActive: Boolean + val isActive: Boolean get() = useCount > 0 - public val isUnconfinedLoopActive: Boolean + val isUnconfinedLoopActive: Boolean get() = useCount >= delta(unconfined = true) // May only be used from the event loop's thread - public val isUnconfinedQueueEmpty: Boolean - get() = unconfinedQueue?.isEmpty ?: true + val isUnconfinedQueueEmpty: Boolean + get() = unconfinedQueue?.isEmpty() ?: true private fun delta(unconfined: Boolean) = if (unconfined) (1L shl 32) else 1L @@ -123,9 +122,8 @@ internal abstract class EventLoop : CoroutineDispatcher() { open fun shutdown() {} } -@ThreadLocal internal object ThreadLocalEventLoop { - private val ref = CommonThreadLocal<EventLoop?>() + private val ref = commonThreadLocal<EventLoop?>(Symbol("ThreadLocalEventLoop")) internal val eventLoop: EventLoop get() = ref.get() ?: createEventLoop().also { ref.set(it) } @@ -142,7 +140,6 @@ internal object ThreadLocalEventLoop { } } -@SharedImmutable private val DISPOSED_TASK = Symbol("REMOVED_TASK") // results for scheduleImpl @@ -168,7 +165,6 @@ internal fun delayToNanos(timeMillis: Long): Long = when { internal fun delayNanosToMillis(timeNanos: Long): Long = timeNanos / MS_TO_NS -@SharedImmutable private val CLOSED_EMPTY = Symbol("CLOSED_EMPTY") private typealias Queue<T> = LockFreeTaskQueueCore<T> @@ -204,7 +200,7 @@ internal abstract class EventLoopImplBase: EventLoopImplPlatform(), Delay { } } - protected override val nextTime: Long + override val nextTime: Long get() { if (super.nextTime == 0L) return 0L val queue = _queue.value @@ -231,7 +227,7 @@ internal abstract class EventLoopImplBase: EventLoopImplPlatform(), Delay { rescheduleAllDelayed() } - public override fun scheduleResumeAfterDelay(timeMillis: Long, continuation: CancellableContinuation<Unit>) { + override fun scheduleResumeAfterDelay(timeMillis: Long, continuation: CancellableContinuation<Unit>) { val timeNanos = delayToNanos(timeMillis) if (timeNanos < MAX_DELAY_NS) { val now = nanoTime() @@ -287,7 +283,7 @@ internal abstract class EventLoopImplBase: EventLoopImplPlatform(), Delay { return nextTime } - public final override fun dispatch(context: CoroutineContext, block: Runnable) = enqueue(block) + final override fun dispatch(context: CoroutineContext, block: Runnable) = enqueue(block) open fun enqueue(task: Runnable) { if (enqueueImpl(task)) { @@ -366,7 +362,7 @@ internal abstract class EventLoopImplBase: EventLoopImplPlatform(), Delay { } - public fun schedule(now: Long, delayedTask: DelayedTask) { + fun schedule(now: Long, delayedTask: DelayedTask) { when (scheduleImpl(now, delayedTask)) { SCHEDULE_OK -> if (shouldUnpark(delayedTask)) unpark() SCHEDULE_COMPLETED -> reschedule(now, delayedTask) @@ -414,7 +410,7 @@ internal abstract class EventLoopImplBase: EventLoopImplPlatform(), Delay { * into heap to avoid overflow and corruption of heap data structure. */ @JvmField var nanoTime: Long - ) : Runnable, Comparable<DelayedTask>, DisposableHandle, ThreadSafeHeapNode { + ) : Runnable, Comparable<DelayedTask>, DisposableHandle, ThreadSafeHeapNode, SynchronizedObject() { @Volatile private var _heap: Any? = null // null | ThreadSafeHeap | DISPOSED_TASK @@ -438,8 +434,7 @@ internal abstract class EventLoopImplBase: EventLoopImplPlatform(), Delay { fun timeToExecute(now: Long): Boolean = now - nanoTime >= 0L - @Synchronized - fun scheduleTask(now: Long, delayed: DelayedTaskQueue, eventLoop: EventLoopImplBase): Int { + fun scheduleTask(now: Long, delayed: DelayedTaskQueue, eventLoop: EventLoopImplBase): Int = synchronized<Int>(this) { if (_heap === DISPOSED_TASK) return SCHEDULE_DISPOSED // don't add -- was already disposed delayed.addLastIf(this) { firstTask -> if (eventLoop.isCompleted) return SCHEDULE_COMPLETED // non-local return from scheduleTask @@ -481,11 +476,9 @@ internal abstract class EventLoopImplBase: EventLoopImplPlatform(), Delay { return SCHEDULE_OK } - @Synchronized - final override fun dispose() { + final override fun dispose(): Unit = synchronized(this) { val heap = _heap if (heap === DISPOSED_TASK) return // already disposed - @Suppress("UNCHECKED_CAST") (heap as? DelayedTaskQueue)?.remove(this) // remove if it is in heap (first) _heap = DISPOSED_TASK // never add again to any heap } @@ -534,7 +527,7 @@ internal expect fun createEventLoop(): EventLoop internal expect fun nanoTime(): Long internal expect object DefaultExecutor { - public fun enqueue(task: Runnable) + fun enqueue(task: Runnable) } /** diff --git a/kotlinx-coroutines-core/common/src/Job.kt b/kotlinx-coroutines-core/common/src/Job.kt index 31d90eee..5f40dfc1 100644 --- a/kotlinx-coroutines-core/common/src/Job.kt +++ b/kotlinx-coroutines-core/common/src/Job.kt @@ -31,7 +31,7 @@ import kotlin.jvm.* * It is completed by calling [CompletableJob.complete]. * * Conceptually, an execution of a job does not produce a result value. Jobs are launched solely for their - * side-effects. See [Deferred] interface for a job that produces a result. + * side effects. See [Deferred] interface for a job that produces a result. * * ### Job states * @@ -118,6 +118,22 @@ public interface Job : CoroutineContext.Element { // ------------ state query ------------ /** + * Returns the parent of the current job if the parent-child relationship + * is established or `null` if the job has no parent or was successfully completed. + * + * Accesses to this property are not idempotent, the property becomes `null` as soon + * as the job is transitioned to its final state, whether it is cancelled or completed, + * and all job children are completed. + * + * For a coroutine, its corresponding job completes as soon as the coroutine itself + * and all its children are complete. + * + * @see [Job] state transitions for additional details. + */ + @ExperimentalCoroutinesApi + public val parent: Job? + + /** * Returns `true` when this job is active -- it was already started and has not completed nor was cancelled yet. * The job that is waiting for its [children] to complete is still considered to be active if it * was not cancelled nor failed. @@ -366,7 +382,6 @@ public interface Job : CoroutineContext.Element { * * If [parent] job is specified, then this job becomes a child job of its parent and * is cancelled when its parent fails or is cancelled. All this job's children are cancelled in this case, too. - * The invocation of [cancel][Job.cancel] with exception (other than [CancellationException]) on this job also cancels parent. * * Conceptually, the resulting job works in the same way as the job created by the `launch { body }` invocation * (see [launch]), but without any code in the body. It is active until cancelled or completed. Invocation of @@ -524,7 +539,7 @@ public fun Job.cancelChildren(cause: Throwable? = null) { /** * Returns `true` when the [Job] of the coroutine in this context is still active - * (has not completed and was not cancelled yet). + * (has not completed and was not cancelled yet) or the context does not have a [Job] in it. * * Check this property in long-running computation loops to support cancellation * when [CoroutineScope.isActive] is not available: @@ -535,11 +550,11 @@ public fun Job.cancelChildren(cause: Throwable? = null) { * } * ``` * - * The `coroutineContext.isActive` expression is a shortcut for `coroutineContext[Job]?.isActive == true`. + * The `coroutineContext.isActive` expression is a shortcut for `get(Job)?.isActive ?: true`. * See [Job.isActive]. */ public val CoroutineContext.isActive: Boolean - get() = this[Job]?.isActive == true + get() = get(Job)?.isActive ?: true /** * Cancels [Job] of this context with an optional cancellation cause. diff --git a/kotlinx-coroutines-core/common/src/JobSupport.kt b/kotlinx-coroutines-core/common/src/JobSupport.kt index 1b5975c8..0c63b6a4 100644 --- a/kotlinx-coroutines-core/common/src/JobSupport.kt +++ b/kotlinx-coroutines-core/common/src/JobSupport.kt @@ -7,13 +7,11 @@ package kotlinx.coroutines import kotlinx.atomicfu.* import kotlinx.coroutines.internal.* -import kotlinx.coroutines.intrinsics.* import kotlinx.coroutines.selects.* import kotlin.coroutines.* import kotlin.coroutines.intrinsics.* import kotlin.js.* import kotlin.jvm.* -import kotlin.native.concurrent.* /** * A concrete implementation of [Job]. It is optionally a child to a parent job. @@ -25,7 +23,7 @@ import kotlin.native.concurrent.* * @suppress **This is unstable API and it is subject to change.** */ @Deprecated(level = DeprecationLevel.ERROR, message = "This is internal API and may be removed in the future releases") -public open class JobSupport constructor(active: Boolean) : Job, ChildJob, ParentJob, SelectClause0 { +public open class JobSupport constructor(active: Boolean) : Job, ChildJob, ParentJob { final override val key: CoroutineContext.Key<*> get() = Job /* @@ -126,6 +124,7 @@ public open class JobSupport constructor(active: Boolean) : Job, ChildJob, Paren */ // Note: use shared objects while we have no listeners + // Used by the IDEA debugger via reflection and must be kept binary-compatible, see KTIJ-24102 private val _state = atomic<Any?>(if (active) EMPTY_ACTIVE else EMPTY_NEW) private val _parentHandle = atomic<ChildHandle?>(null) @@ -133,6 +132,9 @@ public open class JobSupport constructor(active: Boolean) : Job, ChildJob, Paren get() = _parentHandle.value set(value) { _parentHandle.value = value } + override val parent: Job? + get() = parentHandle?.parent + // ------------ initialization ------------ /** @@ -207,7 +209,7 @@ public open class JobSupport constructor(active: Boolean) : Job, ChildJob, Paren assert { state.isCompleting } // consistency check -- must be marked as completing val proposedException = (proposedUpdate as? CompletedExceptionally)?.cause // Create the final exception and seal the state so that no more exceptions can be added - var wasCancelling = false // KLUDGE: we cannot have contract for our own expect fun synchronized + val wasCancelling: Boolean val finalException = synchronized(state) { wasCancelling = state.isCancelling val exceptions = state.sealLocked(proposedException) @@ -559,26 +561,28 @@ public open class JobSupport constructor(active: Boolean) : Job, ChildJob, Paren cont.disposeOnCancellation(invokeOnCompletion(handler = ResumeOnCompletion(cont).asHandler)) } + @Suppress("UNCHECKED_CAST") public final override val onJoin: SelectClause0 - get() = this + get() = SelectClause0Impl( + clauseObject = this@JobSupport, + regFunc = JobSupport::registerSelectForOnJoin as RegistrationFunction + ) - // registerSelectJoin - public final override fun <R> registerSelectClause0(select: SelectInstance<R>, block: suspend () -> R) { - // fast-path -- check state and select/return if needed - loopOnState { state -> - if (select.isSelected) return - if (state !is Incomplete) { - // already complete -- select result - if (select.trySelect()) { - block.startCoroutineUnintercepted(select.completion) - } - return - } - if (startInternal(state) == 0) { - // slow-path -- register waiter for completion - select.disposeOnSelect(invokeOnCompletion(handler = SelectJoinOnCompletion(select, block).asHandler)) - return - } + @Suppress("UNUSED_PARAMETER") + private fun registerSelectForOnJoin(select: SelectInstance<*>, ignoredParam: Any?) { + if (!joinInternal()) { + select.selectInRegistrationPhase(Unit) + return + } + val disposableHandle = invokeOnCompletion(SelectOnJoinCompletionHandler(select).asHandler) + select.disposeOnCompletion(disposableHandle) + } + + private inner class SelectOnJoinCompletionHandler( + private val select: SelectInstance<*> + ) : JobNode() { + override fun invoke(cause: Throwable?) { + select.trySelect(this@JobSupport, Unit) } } @@ -1204,7 +1208,7 @@ public open class JobSupport constructor(active: Boolean) : Job, ChildJob, Paren /** * @suppress **This is unstable API and it is subject to change.** */ - internal suspend fun awaitInternal(): Any? { + protected suspend fun awaitInternal(): Any? { // fast-path -- check state (avoid extra object creation) while (true) { // lock-free loop on state val state = this.state @@ -1234,46 +1238,42 @@ public open class JobSupport constructor(active: Boolean) : Job, ChildJob, Paren cont.getResult() } - /** - * @suppress **This is unstable API and it is subject to change.** - */ - // registerSelectAwaitInternal @Suppress("UNCHECKED_CAST") - internal fun <T, R> registerSelectClause1Internal(select: SelectInstance<R>, block: suspend (T) -> R) { - // fast-path -- check state and select/return if needed - loopOnState { state -> - if (select.isSelected) return + protected val onAwaitInternal: SelectClause1<*> get() = SelectClause1Impl<Any?>( + clauseObject = this@JobSupport, + regFunc = JobSupport::onAwaitInternalRegFunc as RegistrationFunction, + processResFunc = JobSupport::onAwaitInternalProcessResFunc as ProcessResultFunction + ) + + @Suppress("UNUSED_PARAMETER") + private fun onAwaitInternalRegFunc(select: SelectInstance<*>, ignoredParam: Any?) { + while (true) { + val state = this.state if (state !is Incomplete) { - // already complete -- select result - if (select.trySelect()) { - if (state is CompletedExceptionally) { - select.resumeSelectWithException(state.cause) - } - else { - block.startCoroutineUnintercepted(state.unboxState() as T, select.completion) - } - } - return - } - if (startInternal(state) == 0) { - // slow-path -- register waiter for completion - select.disposeOnSelect(invokeOnCompletion(handler = SelectAwaitOnCompletion(select, block).asHandler)) + val result = if (state is CompletedExceptionally) state else state.unboxState() + select.selectInRegistrationPhase(result) return } + if (startInternal(state) >= 0) break // break unless needs to retry } + val disposableHandle = invokeOnCompletion(SelectOnAwaitCompletionHandler(select).asHandler) + select.disposeOnCompletion(disposableHandle) } - /** - * @suppress **This is unstable API and it is subject to change.** - */ - @Suppress("UNCHECKED_CAST") - internal fun <T, R> selectAwaitCompletion(select: SelectInstance<R>, block: suspend (T) -> R) { - val state = this.state - // Note: await is non-atomic (can be cancelled while dispatched) - if (state is CompletedExceptionally) - select.resumeSelectWithException(state.cause) - else - block.startCoroutineCancellable(state.unboxState() as T, select.completion) + @Suppress("UNUSED_PARAMETER") + private fun onAwaitInternalProcessResFunc(ignoredParam: Any?, result: Any?): Any? { + if (result is CompletedExceptionally) throw result.cause + return result + } + + private inner class SelectOnAwaitCompletionHandler( + private val select: SelectInstance<*> + ) : JobNode() { + override fun invoke(cause: Throwable?) { + val state = this@JobSupport.state + val result = if (state is CompletedExceptionally) state else state.unboxState() + select.trySelect(this@JobSupport, result) + } } } @@ -1286,25 +1286,18 @@ internal fun Any?.unboxState(): Any? = (this as? IncompleteStateBox)?.state ?: t // --------------- helper classes & constants for job implementation -@SharedImmutable private val COMPLETING_ALREADY = Symbol("COMPLETING_ALREADY") @JvmField -@SharedImmutable internal val COMPLETING_WAITING_CHILDREN = Symbol("COMPLETING_WAITING_CHILDREN") -@SharedImmutable private val COMPLETING_RETRY = Symbol("COMPLETING_RETRY") -@SharedImmutable private val TOO_LATE_TO_CANCEL = Symbol("TOO_LATE_TO_CANCEL") private const val RETRY = -1 private const val FALSE = 0 private const val TRUE = 1 -@SharedImmutable private val SEALED = Symbol("SEALED") -@SharedImmutable private val EMPTY_NEW = Empty(false) -@SharedImmutable private val EMPTY_ACTIVE = Empty(true) private class Empty(override val isActive: Boolean) : Incomplete { @@ -1421,26 +1414,6 @@ internal class DisposeOnCompletion( override fun invoke(cause: Throwable?) = handle.dispose() } -private class SelectJoinOnCompletion<R>( - private val select: SelectInstance<R>, - private val block: suspend () -> R -) : JobNode() { - override fun invoke(cause: Throwable?) { - if (select.trySelect()) - block.startCoroutineCancellable(select.completion) - } -} - -private class SelectAwaitOnCompletion<T, R>( - private val select: SelectInstance<R>, - private val block: suspend (T) -> R -) : JobNode() { - override fun invoke(cause: Throwable?) { - if (select.trySelect()) - job.selectAwaitCompletion(select, block) - } -} - // -------- invokeOnCancellation nodes /** @@ -1468,7 +1441,9 @@ internal class ChildHandleNode( } // Same as ChildHandleNode, but for cancellable continuation +@PublishedApi internal class ChildContinuation( + // Used by the IDEA debugger via reflection and must be kept binary-compatible, see KTIJ-24102 @JvmField val child: CancellableContinuationImpl<*> ) : JobCancellingNode() { override fun invoke(cause: Throwable?) { diff --git a/kotlinx-coroutines-core/common/src/MainCoroutineDispatcher.kt b/kotlinx-coroutines-core/common/src/MainCoroutineDispatcher.kt index a7065ccd..9150bd06 100644 --- a/kotlinx-coroutines-core/common/src/MainCoroutineDispatcher.kt +++ b/kotlinx-coroutines-core/common/src/MainCoroutineDispatcher.kt @@ -20,7 +20,7 @@ public abstract class MainCoroutineDispatcher : CoroutineDispatcher() { * * Immediate dispatcher is safe from stack overflows and in case of nested invocations forms event-loop similar to [Dispatchers.Unconfined]. * The event loop is an advanced topic and its implications can be found in [Dispatchers.Unconfined] documentation. - * The formed event-loop is shared with [Unconfined] and other immediate dispatchers, potentially overlapping tasks between them. + * The formed event-loop is shared with [Dispatchers.Unconfined] and other immediate dispatchers, potentially overlapping tasks between them. * * Example of usage: * ``` diff --git a/kotlinx-coroutines-core/common/src/NonCancellable.kt b/kotlinx-coroutines-core/common/src/NonCancellable.kt index c2781092..9fb72ddd 100644 --- a/kotlinx-coroutines-core/common/src/NonCancellable.kt +++ b/kotlinx-coroutines-core/common/src/NonCancellable.kt @@ -30,6 +30,14 @@ public object NonCancellable : AbstractCoroutineContextElement(Job), Job { private const val message = "NonCancellable can be used only as an argument for 'withContext', direct usages of its API are prohibited" /** + * Always returns `null`. + * @suppress **This an internal API and should not be used from general code.** + */ + @Deprecated(level = DeprecationLevel.WARNING, message = message) + override val parent: Job? + get() = null + + /** * Always returns `true`. * @suppress **This an internal API and should not be used from general code.** */ diff --git a/kotlinx-coroutines-core/common/src/SchedulerTask.common.kt b/kotlinx-coroutines-core/common/src/SchedulerTask.common.kt index 45e2542b..ae3ffcce 100644 --- a/kotlinx-coroutines-core/common/src/SchedulerTask.common.kt +++ b/kotlinx-coroutines-core/common/src/SchedulerTask.common.kt @@ -4,7 +4,7 @@ package kotlinx.coroutines -internal expect abstract class SchedulerTask() : Runnable +internal expect abstract class SchedulerTask internal constructor() : Runnable internal expect interface SchedulerTaskContext diff --git a/kotlinx-coroutines-core/common/src/Timeout.kt b/kotlinx-coroutines-core/common/src/Timeout.kt index 6a6829df..3ce74c00 100644 --- a/kotlinx-coroutines-core/common/src/Timeout.kt +++ b/kotlinx-coroutines-core/common/src/Timeout.kt @@ -13,10 +13,12 @@ import kotlin.coroutines.* import kotlin.coroutines.intrinsics.* import kotlin.jvm.* import kotlin.time.* +import kotlin.time.Duration.Companion.milliseconds /** * Runs a given suspending [block] of code inside a coroutine with a specified [timeout][timeMillis] and throws * a [TimeoutCancellationException] if the timeout was exceeded. + * If the given [timeMillis] is non-positive, [TimeoutCancellationException] is thrown immediately. * * The code that is executing inside the [block] is cancelled on timeout and the active or next invocation of * the cancellable suspending function inside the block throws a [TimeoutCancellationException]. @@ -25,8 +27,8 @@ import kotlin.time.* * Note that the timeout action can be specified for a [select] invocation with [onTimeout][SelectBuilder.onTimeout] clause. * * **The timeout event is asynchronous with respect to the code running in the block** and may happen at any time, - * even right before the return from inside of the timeout [block]. Keep this in mind if you open or acquire some - * resource inside the [block] that needs closing or release outside of the block. + * even right before the return from inside the timeout [block]. Keep this in mind if you open or acquire some + * resource inside the [block] that needs closing or release outside the block. * See the * [Asynchronous timeout and resources][https://kotlinlang.org/docs/reference/coroutines/cancellation-and-timeouts.html#asynchronous-timeout-and-resources] * section of the coroutines guide for details. @@ -48,6 +50,7 @@ public suspend fun <T> withTimeout(timeMillis: Long, block: suspend CoroutineSco /** * Runs a given suspending [block] of code inside a coroutine with the specified [timeout] and throws * a [TimeoutCancellationException] if the timeout was exceeded. + * If the given [timeout] is non-positive, [TimeoutCancellationException] is thrown immediately. * * The code that is executing inside the [block] is cancelled on timeout and the active or next invocation of * the cancellable suspending function inside the block throws a [TimeoutCancellationException]. @@ -56,8 +59,8 @@ public suspend fun <T> withTimeout(timeMillis: Long, block: suspend CoroutineSco * Note that the timeout action can be specified for a [select] invocation with [onTimeout][SelectBuilder.onTimeout] clause. * * **The timeout event is asynchronous with respect to the code running in the block** and may happen at any time, - * even right before the return from inside of the timeout [block]. Keep this in mind if you open or acquire some - * resource inside the [block] that needs closing or release outside of the block. + * even right before the return from inside the timeout [block]. Keep this in mind if you open or acquire some + * resource inside the [block] that needs closing or release outside the block. * See the * [Asynchronous timeout and resources][https://kotlinlang.org/docs/reference/coroutines/cancellation-and-timeouts.html#asynchronous-timeout-and-resources] * section of the coroutines guide for details. @@ -74,6 +77,7 @@ public suspend fun <T> withTimeout(timeout: Duration, block: suspend CoroutineSc /** * Runs a given suspending block of code inside a coroutine with a specified [timeout][timeMillis] and returns * `null` if this timeout was exceeded. + * If the given [timeMillis] is non-positive, `null` is returned immediately. * * The code that is executing inside the [block] is cancelled on timeout and the active or next invocation of * cancellable suspending function inside the block throws a [TimeoutCancellationException]. @@ -82,8 +86,8 @@ public suspend fun <T> withTimeout(timeout: Duration, block: suspend CoroutineSc * Note that the timeout action can be specified for a [select] invocation with [onTimeout][SelectBuilder.onTimeout] clause. * * **The timeout event is asynchronous with respect to the code running in the block** and may happen at any time, - * even right before the return from inside of the timeout [block]. Keep this in mind if you open or acquire some - * resource inside the [block] that needs closing or release outside of the block. + * even right before the return from inside the timeout [block]. Keep this in mind if you open or acquire some + * resource inside the [block] that needs closing or release outside the block. * See the * [Asynchronous timeout and resources][https://kotlinlang.org/docs/reference/coroutines/cancellation-and-timeouts.html#asynchronous-timeout-and-resources] * section of the coroutines guide for details. @@ -114,6 +118,7 @@ public suspend fun <T> withTimeoutOrNull(timeMillis: Long, block: suspend Corout /** * Runs a given suspending block of code inside a coroutine with the specified [timeout] and returns * `null` if this timeout was exceeded. + * If the given [timeout] is non-positive, `null` is returned immediately. * * The code that is executing inside the [block] is cancelled on timeout and the active or next invocation of * cancellable suspending function inside the block throws a [TimeoutCancellationException]. @@ -122,8 +127,8 @@ public suspend fun <T> withTimeoutOrNull(timeMillis: Long, block: suspend Corout * Note that the timeout action can be specified for a [select] invocation with [onTimeout][SelectBuilder.onTimeout] clause. * * **The timeout event is asynchronous with respect to the code running in the block** and may happen at any time, - * even right before the return from inside of the timeout [block]. Keep this in mind if you open or acquire some - * resource inside the [block] that needs closing or release outside of the block. + * even right before the return from inside the timeout [block]. Keep this in mind if you open or acquire some + * resource inside the [block] that needs closing or release outside the block. * See the * [Asynchronous timeout and resources][https://kotlinlang.org/docs/reference/coroutines/cancellation-and-timeouts.html#asynchronous-timeout-and-resources] * section of the coroutines guide for details. @@ -131,9 +136,9 @@ public suspend fun <T> withTimeoutOrNull(timeMillis: Long, block: suspend Corout * > Implementation note: how the time is tracked exactly is an implementation detail of the context's [CoroutineDispatcher]. */ public suspend fun <T> withTimeoutOrNull(timeout: Duration, block: suspend CoroutineScope.() -> T): T? = - withTimeoutOrNull(timeout.toDelayMillis(), block) + withTimeoutOrNull(timeout.toDelayMillis(), block) -private fun <U, T: U> setupTimeout( +private fun <U, T : U> setupTimeout( coroutine: TimeoutCoroutine<U, T>, block: suspend CoroutineScope.() -> T ): Any? { @@ -146,12 +151,12 @@ private fun <U, T: U> setupTimeout( return coroutine.startUndispatchedOrReturnIgnoreTimeout(coroutine, block) } -private class TimeoutCoroutine<U, in T: U>( +private class TimeoutCoroutine<U, in T : U>( @JvmField val time: Long, uCont: Continuation<U> // unintercepted continuation ) : ScopeCoroutine<T>(uCont.context, uCont), Runnable { override fun run() { - cancelCoroutine(TimeoutCancellationException(time, this)) + cancelCoroutine(TimeoutCancellationException(time, context.delay, this)) } override fun nameString(): String = @@ -169,7 +174,6 @@ public class TimeoutCancellationException internal constructor( * Creates a timeout exception with the given message. * This constructor is needed for exception stack-traces recovery. */ - @Suppress("UNUSED") internal constructor(message: String) : this(message, null) // message is never null in fact @@ -177,8 +181,12 @@ public class TimeoutCancellationException internal constructor( TimeoutCancellationException(message ?: "", coroutine).also { it.initCause(this) } } -@Suppress("FunctionName") internal fun TimeoutCancellationException( time: Long, + delay: Delay, coroutine: Job -) : TimeoutCancellationException = TimeoutCancellationException("Timed out waiting for $time ms", coroutine) +) : TimeoutCancellationException { + val message = (delay as? DelayWithTimeoutDiagnostics)?.timeoutMessage(time.milliseconds) + ?: "Timed out waiting for $time ms" + return TimeoutCancellationException(message, coroutine) +} diff --git a/kotlinx-coroutines-core/common/src/Waiter.kt b/kotlinx-coroutines-core/common/src/Waiter.kt new file mode 100644 index 00000000..79d3dbf5 --- /dev/null +++ b/kotlinx-coroutines-core/common/src/Waiter.kt @@ -0,0 +1,21 @@ +/* + * Copyright 2016-2023 JetBrains s.r.o. Use of this source code is governed by the Apache 2.0 license. + */ + +package kotlinx.coroutines + +import kotlinx.coroutines.internal.Segment +import kotlinx.coroutines.selects.* + +/** + * All waiters (such as [CancellableContinuationImpl] and [SelectInstance]) in synchronization and + * communication primitives, should implement this interface to make the code faster and easier to read. + */ +internal interface Waiter { + /** + * When this waiter is cancelled, [Segment.onCancellation] with + * the specified [segment] and [index] should be called. + * This function installs the corresponding cancellation handler. + */ + fun invokeOnCancellation(segment: Segment<*>, index: Int) +} diff --git a/kotlinx-coroutines-core/common/src/Yield.kt b/kotlinx-coroutines-core/common/src/Yield.kt index 98e21041..db3bfa53 100644 --- a/kotlinx-coroutines-core/common/src/Yield.kt +++ b/kotlinx-coroutines-core/common/src/Yield.kt @@ -5,7 +5,6 @@ package kotlinx.coroutines import kotlinx.coroutines.internal.* -import kotlin.coroutines.* import kotlin.coroutines.intrinsics.* /** diff --git a/kotlinx-coroutines-core/common/src/channels/AbstractChannel.kt b/kotlinx-coroutines-core/common/src/channels/AbstractChannel.kt deleted file mode 100644 index b92ced6a..00000000 --- a/kotlinx-coroutines-core/common/src/channels/AbstractChannel.kt +++ /dev/null @@ -1,1131 +0,0 @@ -/* - * Copyright 2016-2021 JetBrains s.r.o. Use of this source code is governed by the Apache 2.0 license. - */ - -package kotlinx.coroutines.channels - -import kotlinx.atomicfu.* -import kotlinx.coroutines.* -import kotlinx.coroutines.internal.* -import kotlinx.coroutines.intrinsics.* -import kotlinx.coroutines.selects.* -import kotlin.coroutines.* -import kotlin.jvm.* -import kotlin.native.concurrent.* - -/** - * Abstract send channel. It is a base class for all send channel implementations. - */ -internal abstract class AbstractSendChannel<E>( - @JvmField protected val onUndeliveredElement: OnUndeliveredElement<E>? -) : SendChannel<E> { - /** @suppress **This is unstable API and it is subject to change.** */ - protected val queue = LockFreeLinkedListHead() - - // ------ extension points for buffered channels ------ - - /** - * Returns `true` if [isBufferFull] is always `true`. - * @suppress **This is unstable API and it is subject to change.** - */ - protected abstract val isBufferAlwaysFull: Boolean - - /** - * Returns `true` if this channel's buffer is full. - * This operation should be atomic if it is invoked by [enqueueSend]. - * @suppress **This is unstable API and it is subject to change.** - */ - protected abstract val isBufferFull: Boolean - - // State transitions: null -> handler -> HANDLER_INVOKED - private val onCloseHandler = atomic<Any?>(null) - - // ------ internal functions for override by buffered channels ------ - - /** - * Tries to add element to buffer or to queued receiver. - * Return type is `OFFER_SUCCESS | OFFER_FAILED | Closed`. - * @suppress **This is unstable API and it is subject to change.** - */ - protected open fun offerInternal(element: E): Any { - while (true) { - val receive = takeFirstReceiveOrPeekClosed() ?: return OFFER_FAILED - val token = receive.tryResumeReceive(element, null) - if (token != null) { - assert { token === RESUME_TOKEN } - receive.completeResumeReceive(element) - return receive.offerResult - } - } - } - - /** - * Tries to add element to buffer or to queued receiver if select statement clause was not selected yet. - * Return type is `ALREADY_SELECTED | OFFER_SUCCESS | OFFER_FAILED | RETRY_ATOMIC | Closed`. - * @suppress **This is unstable API and it is subject to change.** - */ - protected open fun offerSelectInternal(element: E, select: SelectInstance<*>): Any { - // offer atomically with select - val offerOp = describeTryOffer(element) - val failure = select.performAtomicTrySelect(offerOp) - if (failure != null) return failure - val receive = offerOp.result - receive.completeResumeReceive(element) - return receive.offerResult - } - - // ------ state functions & helpers for concrete implementations ------ - - /** - * Returns non-null closed token if it is last in the queue. - * @suppress **This is unstable API and it is subject to change.** - */ - protected val closedForSend: Closed<*>? get() = (queue.prevNode as? Closed<*>)?.also { helpClose(it) } - - /** - * Returns non-null closed token if it is first in the queue. - * @suppress **This is unstable API and it is subject to change.** - */ - protected val closedForReceive: Closed<*>? get() = (queue.nextNode as? Closed<*>)?.also { helpClose(it) } - - /** - * Retrieves first sending waiter from the queue or returns closed token. - * @suppress **This is unstable API and it is subject to change.** - */ - protected fun takeFirstSendOrPeekClosed(): Send? = - queue.removeFirstIfIsInstanceOfOrPeekIf<Send> { it is Closed<*> } - - /** - * Queues buffered element, returns null on success or - * returns node reference if it was already closed or is waiting for receive. - * @suppress **This is unstable API and it is subject to change.** - */ - protected fun sendBuffered(element: E): ReceiveOrClosed<*>? { - queue.addLastIfPrev(SendBuffered(element)) { prev -> - if (prev is ReceiveOrClosed<*>) return@sendBuffered prev - true - } - return null - } - - /** - * @suppress **This is unstable API and it is subject to change.** - */ - protected fun describeSendBuffered(element: E): AddLastDesc<*> = SendBufferedDesc(queue, element) - - private open class SendBufferedDesc<E>( - queue: LockFreeLinkedListHead, - element: E - ) : AddLastDesc<SendBuffered<E>>(queue, SendBuffered(element)) { - override fun failure(affected: LockFreeLinkedListNode): Any? = when (affected) { - is Closed<*> -> affected - is ReceiveOrClosed<*> -> OFFER_FAILED - else -> null - } - } - - // ------ SendChannel ------ - - public final override val isClosedForSend: Boolean get() = closedForSend != null - private val isFullImpl: Boolean get() = queue.nextNode !is ReceiveOrClosed<*> && isBufferFull - - public final override suspend fun send(element: E) { - // fast path -- try offer non-blocking - if (offerInternal(element) === OFFER_SUCCESS) return - // slow-path does suspend or throws exception - return sendSuspend(element) - } - - @Suppress("DEPRECATION", "DEPRECATION_ERROR") - override fun offer(element: E): Boolean { - // Temporary migration for offer users who rely on onUndeliveredElement - try { - return super.offer(element) - } catch (e: Throwable) { - onUndeliveredElement?.callUndeliveredElementCatchingException(element)?.let { - // If it crashes, add send exception as suppressed for better diagnostics - it.addSuppressed(e) - throw it - } - throw e - } - } - - public final override fun trySend(element: E): ChannelResult<Unit> { - val result = offerInternal(element) - return when { - result === OFFER_SUCCESS -> ChannelResult.success(Unit) - result === OFFER_FAILED -> { - // We should check for closed token on trySend as well, otherwise trySend won't be linearizable - // in the face of concurrent close() - // See https://github.com/Kotlin/kotlinx.coroutines/issues/359 - val closedForSend = closedForSend ?: return ChannelResult.failure() - ChannelResult.closed(helpCloseAndGetSendException(closedForSend)) - } - result is Closed<*> -> { - ChannelResult.closed(helpCloseAndGetSendException(result)) - } - else -> error("trySend returned $result") - } - } - - private fun helpCloseAndGetSendException(closed: Closed<*>): Throwable { - helpClose(closed) - return closed.sendException - } - - private fun helpCloseAndGetSendException(element: E, closed: Closed<*>): Throwable { - // To ensure linearizablity we must ALWAYS help close the channel when we observe that it was closed - // See https://github.com/Kotlin/kotlinx.coroutines/issues/1419 - helpClose(closed) - // Element was not delivered -> cals onUndeliveredElement - onUndeliveredElement?.callUndeliveredElementCatchingException(element)?.let { - // If it crashes, add send exception as suppressed for better diagnostics - it.addSuppressed(closed.sendException) - throw it - } - return closed.sendException - } - - private suspend fun sendSuspend(element: E): Unit = suspendCancellableCoroutineReusable sc@ { cont -> - loop@ while (true) { - if (isFullImpl) { - val send = if (onUndeliveredElement == null) - SendElement(element, cont) else - SendElementWithUndeliveredHandler(element, cont, onUndeliveredElement) - val enqueueResult = enqueueSend(send) - when { - enqueueResult == null -> { // enqueued successfully - cont.removeOnCancellation(send) - return@sc - } - enqueueResult is Closed<*> -> { - cont.helpCloseAndResumeWithSendException(element, enqueueResult) - return@sc - } - enqueueResult === ENQUEUE_FAILED -> {} // try to offer instead - enqueueResult is Receive<*> -> {} // try to offer instead - else -> error("enqueueSend returned $enqueueResult") - } - } - // hm... receiver is waiting or buffer is not full. try to offer - val offerResult = offerInternal(element) - when { - offerResult === OFFER_SUCCESS -> { - cont.resume(Unit) - return@sc - } - offerResult === OFFER_FAILED -> continue@loop - offerResult is Closed<*> -> { - cont.helpCloseAndResumeWithSendException(element, offerResult) - return@sc - } - else -> error("offerInternal returned $offerResult") - } - } - } - - private fun Continuation<*>.helpCloseAndResumeWithSendException(element: E, closed: Closed<*>) { - helpClose(closed) - val sendException = closed.sendException - onUndeliveredElement?.callUndeliveredElementCatchingException(element)?.let { - it.addSuppressed(sendException) - resumeWithException(it) - return - } - resumeWithException(sendException) - } - - /** - * Result is: - * * null -- successfully enqueued - * * ENQUEUE_FAILED -- buffer is not full (should not enqueue) - * * ReceiveOrClosed<*> -- receiver is waiting or it is closed (should not enqueue) - */ - protected open fun enqueueSend(send: Send): Any? { - if (isBufferAlwaysFull) { - queue.addLastIfPrev(send) { prev -> - if (prev is ReceiveOrClosed<*>) return@enqueueSend prev - true - } - } else { - if (!queue.addLastIfPrevAndIf(send, { prev -> - if (prev is ReceiveOrClosed<*>) return@enqueueSend prev - true - }, { isBufferFull })) - return ENQUEUE_FAILED - } - return null - } - - public override fun close(cause: Throwable?): Boolean { - val closed = Closed<E>(cause) - /* - * Try to commit close by adding a close token to the end of the queue. - * Successful -> we're now responsible for closing receivers - * Not successful -> help closing pending receivers to maintain invariant - * "if (!close()) next send will throw" - */ - val closeAdded = queue.addLastIfPrev(closed) { it !is Closed<*> } - val actuallyClosed = if (closeAdded) closed else queue.prevNode as Closed<*> - helpClose(actuallyClosed) - if (closeAdded) invokeOnCloseHandler(cause) - return closeAdded // true if we have closed - } - - private fun invokeOnCloseHandler(cause: Throwable?) { - val handler = onCloseHandler.value - if (handler !== null && handler !== HANDLER_INVOKED - && onCloseHandler.compareAndSet(handler, HANDLER_INVOKED)) { - // CAS failed -> concurrent invokeOnClose() invoked handler - @Suppress("UNCHECKED_CAST") - (handler as Handler)(cause) - } - } - - override fun invokeOnClose(handler: Handler) { - // Intricate dance for concurrent invokeOnClose and close calls - if (!onCloseHandler.compareAndSet(null, handler)) { - val value = onCloseHandler.value - if (value === HANDLER_INVOKED) { - throw IllegalStateException("Another handler was already registered and successfully invoked") - } - - throw IllegalStateException("Another handler was already registered: $value") - } else { - val closedToken = closedForSend - if (closedToken != null && onCloseHandler.compareAndSet(handler, HANDLER_INVOKED)) { - // CAS failed -> close() call invoked handler - (handler)(closedToken.closeCause) - } - } - } - - private fun helpClose(closed: Closed<*>) { - /* - * It's important to traverse list from right to left to avoid races with sender. - * Consider channel state: head -> [receive_1] -> [receive_2] -> head - * - T1 calls receive() - * - T2 calls close() - * - T3 calls close() + send(value) - * - * If both will traverse list from left to right, following non-linearizable history is possible: - * [close -> false], [send -> transferred 'value' to receiver] - * - * Another problem with linearizability of close is that we cannot resume closed receives until all - * receivers are removed from the list. - * Consider channel state: head -> [receive_1] -> [receive_2] -> head - * - T1 called receive_2, and will call send() when it's receive call resumes - * - T2 calls close() - * - * Now if T2's close resumes T1's receive_2 then it's receive gets "closed for receive" exception, but - * its subsequent attempt to send successfully rendezvous with receive_1, producing non-linearizable execution. - */ - var closedList = InlineList<Receive<E>>() - while (true) { - // Break when channel is empty or has no receivers - @Suppress("UNCHECKED_CAST") - val previous = closed.prevNode as? Receive<E> ?: break - if (!previous.remove()) { - // failed to remove the node (due to race) -- retry finding non-removed prevNode - // NOTE: remove() DOES NOT help pending remove operation (that marked next pointer) - previous.helpRemove() // make sure remove is complete before continuing - continue - } - // add removed nodes to a separate list - closedList += previous - } - /* - * Now notify all removed nodes that the channel was closed - * in the order they were added to the channel - */ - closedList.forEachReversed { it.resumeReceiveClosed(closed) } - // and do other post-processing - onClosedIdempotent(closed) - } - - /** - * Invoked when channel is closed as the last action of [close] invocation. - * This method should be idempotent and can be called multiple times. - */ - protected open fun onClosedIdempotent(closed: LockFreeLinkedListNode) {} - - /** - * Retrieves first receiving waiter from the queue or returns closed token. - * @suppress **This is unstable API and it is subject to change.** - */ - protected open fun takeFirstReceiveOrPeekClosed(): ReceiveOrClosed<E>? = - queue.removeFirstIfIsInstanceOfOrPeekIf<ReceiveOrClosed<E>>({ it is Closed<*> }) - - // ------ registerSelectSend ------ - - /** - * @suppress **This is unstable API and it is subject to change.** - */ - protected fun describeTryOffer(element: E): TryOfferDesc<E> = TryOfferDesc(element, queue) - - /** - * @suppress **This is unstable API and it is subject to change.** - */ - protected class TryOfferDesc<E>( - @JvmField val element: E, - queue: LockFreeLinkedListHead - ) : RemoveFirstDesc<ReceiveOrClosed<E>>(queue) { - override fun failure(affected: LockFreeLinkedListNode): Any? = when (affected) { - is Closed<*> -> affected - !is ReceiveOrClosed<*> -> OFFER_FAILED - else -> null - } - - @Suppress("UNCHECKED_CAST") - override fun onPrepare(prepareOp: PrepareOp): Any? { - val affected = prepareOp.affected as ReceiveOrClosed<E> // see "failure" impl - val token = affected.tryResumeReceive(element, prepareOp) ?: return REMOVE_PREPARED - if (token === RETRY_ATOMIC) return RETRY_ATOMIC - assert { token === RESUME_TOKEN } - return null - } - } - - final override val onSend: SelectClause2<E, SendChannel<E>> - get() = object : SelectClause2<E, SendChannel<E>> { - override fun <R> registerSelectClause2(select: SelectInstance<R>, param: E, block: suspend (SendChannel<E>) -> R) { - registerSelectSend(select, param, block) - } - } - - private fun <R> registerSelectSend(select: SelectInstance<R>, element: E, block: suspend (SendChannel<E>) -> R) { - while (true) { - if (select.isSelected) return - if (isFullImpl) { - val node = SendSelect(element, this, select, block) - val enqueueResult = enqueueSend(node) - when { - enqueueResult == null -> { // enqueued successfully - select.disposeOnSelect(node) - return - } - enqueueResult is Closed<*> -> throw recoverStackTrace(helpCloseAndGetSendException(element, enqueueResult)) - enqueueResult === ENQUEUE_FAILED -> {} // try to offer - enqueueResult is Receive<*> -> {} // try to offer - else -> error("enqueueSend returned $enqueueResult ") - } - } - // hm... receiver is waiting or buffer is not full. try to offer - val offerResult = offerSelectInternal(element, select) - when { - offerResult === ALREADY_SELECTED -> return - offerResult === OFFER_FAILED -> {} // retry - offerResult === RETRY_ATOMIC -> {} // retry - offerResult === OFFER_SUCCESS -> { - block.startCoroutineUnintercepted(receiver = this, completion = select.completion) - return - } - offerResult is Closed<*> -> throw recoverStackTrace(helpCloseAndGetSendException(element, offerResult)) - else -> error("offerSelectInternal returned $offerResult") - } - } - } - - // ------ debug ------ - - public override fun toString() = - "$classSimpleName@$hexAddress{$queueDebugStateString}$bufferDebugString" - - private val queueDebugStateString: String - get() { - val head = queue.nextNode - if (head === queue) return "EmptyQueue" - var result = when (head) { - is Closed<*> -> head.toString() - is Receive<*> -> "ReceiveQueued" - is Send -> "SendQueued" - else -> "UNEXPECTED:$head" // should not happen - } - val tail = queue.prevNode - if (tail !== head) { - result += ",queueSize=${countQueueSize()}" - if (tail is Closed<*>) result += ",closedForSend=$tail" - } - return result - } - - private fun countQueueSize(): Int { - var size = 0 - queue.forEach<LockFreeLinkedListNode> { size++ } - return size - } - - protected open val bufferDebugString: String get() = "" - - // ------ private ------ - - private class SendSelect<E, R>( - override val pollResult: E, // E | Closed - the result pollInternal returns when it rendezvous with this node - @JvmField val channel: AbstractSendChannel<E>, - @JvmField val select: SelectInstance<R>, - @JvmField val block: suspend (SendChannel<E>) -> R - ) : Send(), DisposableHandle { - override fun tryResumeSend(otherOp: PrepareOp?): Symbol? = - select.trySelectOther(otherOp) as Symbol? // must return symbol - - override fun completeResumeSend() { - block.startCoroutineCancellable(receiver = channel, completion = select.completion) - } - - override fun dispose() { // invoked on select completion - if (!remove()) return - // if the node was successfully removed (meaning it was added but was not received) then element not delivered - undeliveredElement() - } - - override fun resumeSendClosed(closed: Closed<*>) { - if (select.trySelect()) - select.resumeSelectWithException(closed.sendException) - } - - override fun undeliveredElement() { - channel.onUndeliveredElement?.callUndeliveredElement(pollResult, select.completion.context) - } - - override fun toString(): String = "SendSelect@$hexAddress($pollResult)[$channel, $select]" - } - - internal class SendBuffered<out E>( - @JvmField val element: E - ) : Send() { - override val pollResult: Any? get() = element - override fun tryResumeSend(otherOp: PrepareOp?): Symbol? = RESUME_TOKEN.also { otherOp?.finishPrepare() } - override fun completeResumeSend() {} - - /** - * This method should be never called, see special logic in [LinkedListChannel.onCancelIdempotentList]. - */ - override fun resumeSendClosed(closed: Closed<*>) { - assert { false } - } - - override fun toString(): String = "SendBuffered@$hexAddress($element)" - } -} - -/** - * Abstract send/receive channel. It is a base class for all channel implementations. - */ -internal abstract class AbstractChannel<E>( - onUndeliveredElement: OnUndeliveredElement<E>? -) : AbstractSendChannel<E>(onUndeliveredElement), Channel<E> { - // ------ extension points for buffered channels ------ - - /** - * Returns `true` if [isBufferEmpty] is always `true`. - * @suppress **This is unstable API and it is subject to change.** - */ - protected abstract val isBufferAlwaysEmpty: Boolean - - /** - * Returns `true` if this channel's buffer is empty. - * This operation should be atomic if it is invoked by [enqueueReceive]. - * @suppress **This is unstable API and it is subject to change.** - */ - protected abstract val isBufferEmpty: Boolean - - // ------ internal functions for override by buffered channels ------ - - /** - * Tries to remove element from buffer or from queued sender. - * Return type is `E | POLL_FAILED | Closed` - * @suppress **This is unstable API and it is subject to change.** - */ - protected open fun pollInternal(): Any? { - while (true) { - val send = takeFirstSendOrPeekClosed() ?: return POLL_FAILED - val token = send.tryResumeSend(null) - if (token != null) { - assert { token === RESUME_TOKEN } - send.completeResumeSend() - return send.pollResult - } - // too late, already cancelled, but we removed it from the queue and need to notify on undelivered element - send.undeliveredElement() - } - } - - /** - * Tries to remove element from buffer or from queued sender if select statement clause was not selected yet. - * Return type is `ALREADY_SELECTED | E | POLL_FAILED | RETRY_ATOMIC | Closed` - * @suppress **This is unstable API and it is subject to change.** - */ - protected open fun pollSelectInternal(select: SelectInstance<*>): Any? { - // poll atomically with select - val pollOp = describeTryPoll() - val failure = select.performAtomicTrySelect(pollOp) - if (failure != null) return failure - val send = pollOp.result - send.completeResumeSend() - return pollOp.result.pollResult - } - - // ------ state functions & helpers for concrete implementations ------ - - /** - * @suppress **This is unstable API and it is subject to change.** - */ - protected val hasReceiveOrClosed: Boolean get() = queue.nextNode is ReceiveOrClosed<*> - - // ------ ReceiveChannel ------ - - public override val isClosedForReceive: Boolean get() = closedForReceive != null && isBufferEmpty - public override val isEmpty: Boolean get() = isEmptyImpl - protected val isEmptyImpl: Boolean get() = queue.nextNode !is Send && isBufferEmpty - - public final override suspend fun receive(): E { - // fast path -- try poll non-blocking - val result = pollInternal() - /* - * If result is Closed -- go to tail-call slow-path that will allow us to - * properly recover stacktrace without paying a performance cost on fast path. - * We prefer to recover stacktrace using suspending path to have a more precise stacktrace. - */ - @Suppress("UNCHECKED_CAST") - if (result !== POLL_FAILED && result !is Closed<*>) return result as E - // slow-path does suspend - return receiveSuspend(RECEIVE_THROWS_ON_CLOSE) - } - - @Suppress("UNCHECKED_CAST") - private suspend fun <R> receiveSuspend(receiveMode: Int): R = suspendCancellableCoroutineReusable sc@ { cont -> - val receive = if (onUndeliveredElement == null) - ReceiveElement(cont as CancellableContinuation<Any?>, receiveMode) else - ReceiveElementWithUndeliveredHandler(cont as CancellableContinuation<Any?>, receiveMode, onUndeliveredElement) - while (true) { - if (enqueueReceive(receive)) { - removeReceiveOnCancel(cont, receive) - return@sc - } - // hm... something is not right. try to poll - val result = pollInternal() - if (result is Closed<*>) { - receive.resumeReceiveClosed(result) - return@sc - } - if (result !== POLL_FAILED) { - cont.resume(receive.resumeValue(result as E), receive.resumeOnCancellationFun(result as E)) - return@sc - } - } - } - - protected open fun enqueueReceiveInternal(receive: Receive<E>): Boolean = if (isBufferAlwaysEmpty) - queue.addLastIfPrev(receive) { it !is Send } else - queue.addLastIfPrevAndIf(receive, { it !is Send }, { isBufferEmpty }) - - private fun enqueueReceive(receive: Receive<E>) = enqueueReceiveInternal(receive).also { result -> - if (result) onReceiveEnqueued() - } - - @Suppress("UNCHECKED_CAST") - public final override suspend fun receiveCatching(): ChannelResult<E> { - // fast path -- try poll non-blocking - val result = pollInternal() - if (result !== POLL_FAILED) return result.toResult() - // slow-path does suspend - return receiveSuspend(RECEIVE_RESULT) - } - - @Suppress("UNCHECKED_CAST") - public final override fun tryReceive(): ChannelResult<E> { - val result = pollInternal() - if (result === POLL_FAILED) return ChannelResult.failure() - if (result is Closed<*>) return ChannelResult.closed(result.closeCause) - return ChannelResult.success(result as E) - } - - @Deprecated(level = DeprecationLevel.HIDDEN, message = "Since 1.2.0, binary compatibility with versions <= 1.1.x") - final override fun cancel(cause: Throwable?): Boolean = - cancelInternal(cause) - - final override fun cancel(cause: CancellationException?) { - /* - * Do not create an exception if channel is already cancelled. - * Channel is closed for receive when either it is cancelled (then we are free to bail out) - * or was closed and elements were received. - * Then `onCancelIdempotent` does nothing for all implementations. - */ - if (isClosedForReceive) return - cancelInternal(cause ?: CancellationException("$classSimpleName was cancelled")) - } - - // It needs to be internal to support deprecated cancel(Throwable?) API - internal fun cancelInternal(cause: Throwable?): Boolean = - close(cause).also { - onCancelIdempotent(it) - } - - /** - * Method that is invoked right after [close] in [cancel] sequence. - * [wasClosed] is directly mapped to the value returned by [close]. - */ - protected open fun onCancelIdempotent(wasClosed: Boolean) { - /* - * See the comment to helpClose, all these machinery (reversed order of iteration, postponed resume) - * has the same rationale. - */ - val closed = closedForSend ?: error("Cannot happen") - var list = InlineList<Send>() - while (true) { - val previous = closed.prevNode - if (previous is LockFreeLinkedListHead) { - break - } - assert { previous is Send } - if (!previous.remove()) { - previous.helpRemove() // make sure remove is complete before continuing - continue - } - // Add to the list only **after** successful removal - list += previous as Send - } - onCancelIdempotentList(list, closed) - } - - /** - * This method is overridden by [LinkedListChannel] to handle cancellation of [SendBuffered] elements from the list. - */ - protected open fun onCancelIdempotentList(list: InlineList<Send>, closed: Closed<*>) { - list.forEachReversed { it.resumeSendClosed(closed) } - } - - public final override fun iterator(): ChannelIterator<E> = Itr(this) - - // ------ registerSelectReceive ------ - - /** - * @suppress **This is unstable API and it is subject to change.** - */ - protected fun describeTryPoll(): TryPollDesc<E> = TryPollDesc(queue) - - /** - * @suppress **This is unstable API and it is subject to change.** - */ - protected class TryPollDesc<E>(queue: LockFreeLinkedListHead) : RemoveFirstDesc<Send>(queue) { - override fun failure(affected: LockFreeLinkedListNode): Any? = when (affected) { - is Closed<*> -> affected - !is Send -> POLL_FAILED - else -> null - } - - @Suppress("UNCHECKED_CAST") - override fun onPrepare(prepareOp: PrepareOp): Any? { - val affected = prepareOp.affected as Send // see "failure" impl - val token = affected.tryResumeSend(prepareOp) ?: return REMOVE_PREPARED - if (token === RETRY_ATOMIC) return RETRY_ATOMIC - assert { token === RESUME_TOKEN } - return null - } - - override fun onRemoved(affected: LockFreeLinkedListNode) { - // Called when we removed it from the queue but were too late to resume, so we have undelivered element - (affected as Send).undeliveredElement() - } - } - - final override val onReceive: SelectClause1<E> - get() = object : SelectClause1<E> { - @Suppress("UNCHECKED_CAST") - override fun <R> registerSelectClause1(select: SelectInstance<R>, block: suspend (E) -> R) { - registerSelectReceiveMode(select, RECEIVE_THROWS_ON_CLOSE, block as suspend (Any?) -> R) - } - } - - final override val onReceiveCatching: SelectClause1<ChannelResult<E>> - get() = object : SelectClause1<ChannelResult<E>> { - @Suppress("UNCHECKED_CAST") - override fun <R> registerSelectClause1(select: SelectInstance<R>, block: suspend (ChannelResult<E>) -> R) { - registerSelectReceiveMode(select, RECEIVE_RESULT, block as suspend (Any?) -> R) - } - } - - private fun <R> registerSelectReceiveMode(select: SelectInstance<R>, receiveMode: Int, block: suspend (Any?) -> R) { - while (true) { - if (select.isSelected) return - if (isEmptyImpl) { - if (enqueueReceiveSelect(select, block, receiveMode)) return - } else { - val pollResult = pollSelectInternal(select) - when { - pollResult === ALREADY_SELECTED -> return - pollResult === POLL_FAILED -> {} // retry - pollResult === RETRY_ATOMIC -> {} // retry - else -> block.tryStartBlockUnintercepted(select, receiveMode, pollResult) - } - } - } - } - - private fun <R> (suspend (Any?) -> R).tryStartBlockUnintercepted(select: SelectInstance<R>, receiveMode: Int, value: Any?) { - when (value) { - is Closed<*> -> { - when (receiveMode) { - RECEIVE_THROWS_ON_CLOSE -> { - throw recoverStackTrace(value.receiveException) - } - RECEIVE_RESULT -> { - if (!select.trySelect()) return - startCoroutineUnintercepted(ChannelResult.closed<Any>(value.closeCause), select.completion) - } - } - } - else -> { - if (receiveMode == RECEIVE_RESULT) { - startCoroutineUnintercepted(value.toResult<Any>(), select.completion) - } else { - startCoroutineUnintercepted(value, select.completion) - } - } - } - } - - private fun <R> enqueueReceiveSelect( - select: SelectInstance<R>, - block: suspend (Any?) -> R, - receiveMode: Int - ): Boolean { - val node = ReceiveSelect(this, select, block, receiveMode) - val result = enqueueReceive(node) - if (result) select.disposeOnSelect(node) - return result - } - - // ------ protected ------ - - override fun takeFirstReceiveOrPeekClosed(): ReceiveOrClosed<E>? = - super.takeFirstReceiveOrPeekClosed().also { - if (it != null && it !is Closed<*>) onReceiveDequeued() - } - - /** - * Invoked when receiver is successfully enqueued to the queue of waiting receivers. - * @suppress **This is unstable API and it is subject to change.** - */ - protected open fun onReceiveEnqueued() {} - - /** - * Invoked when enqueued receiver was successfully removed from the queue of waiting receivers. - * @suppress **This is unstable API and it is subject to change.** - */ - protected open fun onReceiveDequeued() {} - - // ------ private ------ - - private fun removeReceiveOnCancel(cont: CancellableContinuation<*>, receive: Receive<*>) = - cont.invokeOnCancellation(handler = RemoveReceiveOnCancel(receive).asHandler) - - private inner class RemoveReceiveOnCancel(private val receive: Receive<*>) : BeforeResumeCancelHandler() { - override fun invoke(cause: Throwable?) { - if (receive.remove()) - onReceiveDequeued() - } - override fun toString(): String = "RemoveReceiveOnCancel[$receive]" - } - - private class Itr<E>(@JvmField val channel: AbstractChannel<E>) : ChannelIterator<E> { - var result: Any? = POLL_FAILED // E | POLL_FAILED | Closed - - override suspend fun hasNext(): Boolean { - // check for repeated hasNext - if (result !== POLL_FAILED) return hasNextResult(result) - // fast path -- try poll non-blocking - result = channel.pollInternal() - if (result !== POLL_FAILED) return hasNextResult(result) - // slow-path does suspend - return hasNextSuspend() - } - - private fun hasNextResult(result: Any?): Boolean { - if (result is Closed<*>) { - if (result.closeCause != null) throw recoverStackTrace(result.receiveException) - return false - } - return true - } - - private suspend fun hasNextSuspend(): Boolean = suspendCancellableCoroutineReusable sc@ { cont -> - val receive = ReceiveHasNext(this, cont) - while (true) { - if (channel.enqueueReceive(receive)) { - channel.removeReceiveOnCancel(cont, receive) - return@sc - } - // hm... something is not right. try to poll - val result = channel.pollInternal() - this.result = result - if (result is Closed<*>) { - if (result.closeCause == null) - cont.resume(false) - else - cont.resumeWithException(result.receiveException) - return@sc - } - if (result !== POLL_FAILED) { - @Suppress("UNCHECKED_CAST") - cont.resume(true, channel.onUndeliveredElement?.bindCancellationFun(result as E, cont.context)) - return@sc - } - } - } - - @Suppress("UNCHECKED_CAST") - override fun next(): E { - val result = this.result - if (result is Closed<*>) throw recoverStackTrace(result.receiveException) - if (result !== POLL_FAILED) { - this.result = POLL_FAILED - return result as E - } - - throw IllegalStateException("'hasNext' should be called prior to 'next' invocation") - } - } - - private open class ReceiveElement<in E>( - @JvmField val cont: CancellableContinuation<Any?>, - @JvmField val receiveMode: Int - ) : Receive<E>() { - fun resumeValue(value: E): Any? = when (receiveMode) { - RECEIVE_RESULT -> ChannelResult.success(value) - else -> value - } - - override fun tryResumeReceive(value: E, otherOp: PrepareOp?): Symbol? { - val token = cont.tryResume(resumeValue(value), otherOp?.desc, resumeOnCancellationFun(value)) ?: return null - assert { token === RESUME_TOKEN } // the only other possible result - // We can call finishPrepare only after successful tryResume, so that only good affected node is saved - otherOp?.finishPrepare() - return RESUME_TOKEN - } - - override fun completeResumeReceive(value: E) = cont.completeResume(RESUME_TOKEN) - - override fun resumeReceiveClosed(closed: Closed<*>) { - when { - receiveMode == RECEIVE_RESULT -> cont.resume(closed.toResult<Any>()) - else -> cont.resumeWithException(closed.receiveException) - } - } - override fun toString(): String = "ReceiveElement@$hexAddress[receiveMode=$receiveMode]" - } - - private class ReceiveElementWithUndeliveredHandler<in E>( - cont: CancellableContinuation<Any?>, - receiveMode: Int, - @JvmField val onUndeliveredElement: OnUndeliveredElement<E> - ) : ReceiveElement<E>(cont, receiveMode) { - override fun resumeOnCancellationFun(value: E): ((Throwable) -> Unit)? = - onUndeliveredElement.bindCancellationFun(value, cont.context) - } - - private open class ReceiveHasNext<E>( - @JvmField val iterator: Itr<E>, - @JvmField val cont: CancellableContinuation<Boolean> - ) : Receive<E>() { - override fun tryResumeReceive(value: E, otherOp: PrepareOp?): Symbol? { - val token = cont.tryResume(true, otherOp?.desc, resumeOnCancellationFun(value)) - ?: return null - assert { token === RESUME_TOKEN } // the only other possible result - // We can call finishPrepare only after successful tryResume, so that only good affected node is saved - otherOp?.finishPrepare() - return RESUME_TOKEN - } - - override fun completeResumeReceive(value: E) { - /* - When otherOp != null invocation of tryResumeReceive can happen multiple times and much later, - but completeResumeReceive is called once so we set iterator result here. - */ - iterator.result = value - cont.completeResume(RESUME_TOKEN) - } - - override fun resumeReceiveClosed(closed: Closed<*>) { - val token = if (closed.closeCause == null) { - cont.tryResume(false) - } else { - cont.tryResumeWithException(closed.receiveException) - } - if (token != null) { - iterator.result = closed - cont.completeResume(token) - } - } - - override fun resumeOnCancellationFun(value: E): ((Throwable) -> Unit)? = - iterator.channel.onUndeliveredElement?.bindCancellationFun(value, cont.context) - - override fun toString(): String = "ReceiveHasNext@$hexAddress" - } - - private class ReceiveSelect<R, E>( - @JvmField val channel: AbstractChannel<E>, - @JvmField val select: SelectInstance<R>, - @JvmField val block: suspend (Any?) -> R, - @JvmField val receiveMode: Int - ) : Receive<E>(), DisposableHandle { - override fun tryResumeReceive(value: E, otherOp: PrepareOp?): Symbol? = - select.trySelectOther(otherOp) as Symbol? - - @Suppress("UNCHECKED_CAST") - override fun completeResumeReceive(value: E) { - block.startCoroutineCancellable( - if (receiveMode == RECEIVE_RESULT) ChannelResult.success(value) else value, - select.completion, - resumeOnCancellationFun(value) - ) - } - - override fun resumeReceiveClosed(closed: Closed<*>) { - if (!select.trySelect()) return - when (receiveMode) { - RECEIVE_THROWS_ON_CLOSE -> select.resumeSelectWithException(closed.receiveException) - RECEIVE_RESULT -> block.startCoroutineCancellable(ChannelResult.closed<R>(closed.closeCause), select.completion) - } - } - - override fun dispose() { // invoked on select completion - if (remove()) - channel.onReceiveDequeued() // notify cancellation of receive - } - - override fun resumeOnCancellationFun(value: E): ((Throwable) -> Unit)? = - channel.onUndeliveredElement?.bindCancellationFun(value, select.completion.context) - - override fun toString(): String = "ReceiveSelect@$hexAddress[$select,receiveMode=$receiveMode]" - } -} - -// receiveMode values -internal const val RECEIVE_THROWS_ON_CLOSE = 0 -internal const val RECEIVE_RESULT = 1 - -@JvmField -@SharedImmutable -internal val EMPTY = Symbol("EMPTY") // marker for Conflated & Buffered channels - -@JvmField -@SharedImmutable -internal val OFFER_SUCCESS = Symbol("OFFER_SUCCESS") - -@JvmField -@SharedImmutable -internal val OFFER_FAILED = Symbol("OFFER_FAILED") - -@JvmField -@SharedImmutable -internal val POLL_FAILED = Symbol("POLL_FAILED") - -@JvmField -@SharedImmutable -internal val ENQUEUE_FAILED = Symbol("ENQUEUE_FAILED") - -@JvmField -@SharedImmutable -internal val HANDLER_INVOKED = Symbol("ON_CLOSE_HANDLER_INVOKED") - -internal typealias Handler = (Throwable?) -> Unit - -/** - * Represents sending waiter in the queue. - */ -internal abstract class Send : LockFreeLinkedListNode() { - abstract val pollResult: Any? // E | Closed - the result pollInternal returns when it rendezvous with this node - // Returns: null - failure, - // RETRY_ATOMIC for retry (only when otherOp != null), - // RESUME_TOKEN on success (call completeResumeSend) - // Must call otherOp?.finishPrepare() after deciding on result other than RETRY_ATOMIC - abstract fun tryResumeSend(otherOp: PrepareOp?): Symbol? - abstract fun completeResumeSend() - abstract fun resumeSendClosed(closed: Closed<*>) - open fun undeliveredElement() {} -} - -/** - * Represents receiver waiter in the queue or closed token. - */ -internal interface ReceiveOrClosed<in E> { - val offerResult: Any // OFFER_SUCCESS | Closed - // Returns: null - failure, - // RETRY_ATOMIC for retry (only when otherOp != null), - // RESUME_TOKEN on success (call completeResumeReceive) - // Must call otherOp?.finishPrepare() after deciding on result other than RETRY_ATOMIC - fun tryResumeReceive(value: E, otherOp: PrepareOp?): Symbol? - fun completeResumeReceive(value: E) -} - -/** - * Represents sender for a specific element. - */ -internal open class SendElement<E>( - override val pollResult: E, - @JvmField val cont: CancellableContinuation<Unit> -) : Send() { - override fun tryResumeSend(otherOp: PrepareOp?): Symbol? { - val token = cont.tryResume(Unit, otherOp?.desc) ?: return null - assert { token === RESUME_TOKEN } // the only other possible result - // We can call finishPrepare only after successful tryResume, so that only good affected node is saved - otherOp?.finishPrepare() // finish preparations - return RESUME_TOKEN - } - - override fun completeResumeSend() = cont.completeResume(RESUME_TOKEN) - override fun resumeSendClosed(closed: Closed<*>) = cont.resumeWithException(closed.sendException) - override fun toString(): String = "$classSimpleName@$hexAddress($pollResult)" -} - -internal class SendElementWithUndeliveredHandler<E>( - pollResult: E, - cont: CancellableContinuation<Unit>, - @JvmField val onUndeliveredElement: OnUndeliveredElement<E> -) : SendElement<E>(pollResult, cont) { - override fun remove(): Boolean { - if (!super.remove()) return false - // if the node was successfully removed (meaning it was added but was not received) then we have undelivered element - undeliveredElement() - return true - } - - override fun undeliveredElement() { - onUndeliveredElement.callUndeliveredElement(pollResult, cont.context) - } -} - -/** - * Represents closed channel. - */ -internal class Closed<in E>( - @JvmField val closeCause: Throwable? -) : Send(), ReceiveOrClosed<E> { - val sendException: Throwable get() = closeCause ?: ClosedSendChannelException(DEFAULT_CLOSE_MESSAGE) - val receiveException: Throwable get() = closeCause ?: ClosedReceiveChannelException(DEFAULT_CLOSE_MESSAGE) - - override val offerResult get() = this - override val pollResult get() = this - override fun tryResumeSend(otherOp: PrepareOp?): Symbol = RESUME_TOKEN.also { otherOp?.finishPrepare() } - override fun completeResumeSend() {} - override fun tryResumeReceive(value: E, otherOp: PrepareOp?): Symbol = RESUME_TOKEN.also { otherOp?.finishPrepare() } - override fun completeResumeReceive(value: E) {} - override fun resumeSendClosed(closed: Closed<*>) = assert { false } // "Should be never invoked" - override fun toString(): String = "Closed@$hexAddress[$closeCause]" -} - -internal abstract class Receive<in E> : LockFreeLinkedListNode(), ReceiveOrClosed<E> { - override val offerResult get() = OFFER_SUCCESS - abstract fun resumeReceiveClosed(closed: Closed<*>) - open fun resumeOnCancellationFun(value: E): ((Throwable) -> Unit)? = null -} - -@Suppress("NOTHING_TO_INLINE", "UNCHECKED_CAST") -private inline fun <E> Any?.toResult(): ChannelResult<E> = - if (this is Closed<*>) ChannelResult.closed(closeCause) else ChannelResult.success(this as E) - -@Suppress("NOTHING_TO_INLINE") -private inline fun <E> Closed<*>.toResult(): ChannelResult<E> = ChannelResult.closed(closeCause) diff --git a/kotlinx-coroutines-core/common/src/channels/ArrayBroadcastChannel.kt b/kotlinx-coroutines-core/common/src/channels/ArrayBroadcastChannel.kt deleted file mode 100644 index 0a96f753..00000000 --- a/kotlinx-coroutines-core/common/src/channels/ArrayBroadcastChannel.kt +++ /dev/null @@ -1,384 +0,0 @@ -/* - * Copyright 2016-2021 JetBrains s.r.o. Use of this source code is governed by the Apache 2.0 license. - */ - -package kotlinx.coroutines.channels - -import kotlinx.atomicfu.* -import kotlinx.coroutines.* -import kotlinx.coroutines.internal.* -import kotlinx.coroutines.selects.* - -/** - * Broadcast channel with array buffer of a fixed [capacity]. - * Sender suspends only when buffer is full due to one of the receives being slow to consume and - * receiver suspends only when buffer is empty. - * - * **Note**, that elements that are sent to this channel while there are no - * [openSubscription] subscribers are immediately lost. - * - * This channel is created by `BroadcastChannel(capacity)` factory function invocation. - * - * This implementation uses lock to protect the buffer, which is held only during very short buffer-update operations. - * The lock at each subscription is also used to manage concurrent attempts to receive from the same subscriber. - * The lists of suspended senders or receivers are lock-free. - */ -internal class ArrayBroadcastChannel<E>( - /** - * Buffer capacity. - */ - val capacity: Int -) : AbstractSendChannel<E>(null), BroadcastChannel<E> { - init { - require(capacity >= 1) { "ArrayBroadcastChannel capacity must be at least 1, but $capacity was specified" } - } - - /** - * NB: prior to changing any logic of ArrayBroadcastChannel internals, please ensure that - * you do not break internal invariants of the SubscriberList implementation on K/N and KJS - */ - - /* - * Writes to buffer are guarded by bufferLock, but reads from buffer are concurrent with writes - * - Write element to buffer then write "tail" (volatile) - * - Read "tail" (volatile), then read element from buffer - * So read/writes to buffer need not be volatile - */ - private val bufferLock = ReentrantLock() - private val buffer = arrayOfNulls<Any?>(capacity) - - // head & tail are Long (64 bits) and we assume that they never wrap around - // head, tail, and size are guarded by bufferLock - - private val _head = atomic(0L) - private var head: Long // do modulo on use of head - get() = _head.value - set(value) { _head.value = value } - - private val _tail = atomic(0L) - private var tail: Long // do modulo on use of tail - get() = _tail.value - set(value) { _tail.value = value } - - private val _size = atomic(0) - private var size: Int - get() = _size.value - set(value) { _size.value = value } - - @Suppress("DEPRECATION") - private val subscribers = subscriberList<Subscriber<E>>() - - override val isBufferAlwaysFull: Boolean get() = false - override val isBufferFull: Boolean get() = size >= capacity - - public override fun openSubscription(): ReceiveChannel<E> = - Subscriber(this).also { - updateHead(addSub = it) - } - - public override fun close(cause: Throwable?): Boolean { - if (!super.close(cause)) return false - checkSubOffers() - return true - } - - @Deprecated(level = DeprecationLevel.HIDDEN, message = "Since 1.2.0, binary compatibility with versions <= 1.1.x") - override fun cancel(cause: Throwable?): Boolean = - cancelInternal(cause) - - override fun cancel(cause: CancellationException?) { - cancelInternal(cause) - } - - private fun cancelInternal(cause: Throwable?): Boolean = - close(cause).also { - for (sub in subscribers) sub.cancelInternal(cause) - } - - // result is `OFFER_SUCCESS | OFFER_FAILED | Closed` - override fun offerInternal(element: E): Any { - bufferLock.withLock { - // check if closed for send (under lock, so size cannot change) - closedForSend?.let { return it } - val size = this.size - if (size >= capacity) return OFFER_FAILED - val tail = this.tail - buffer[(tail % capacity).toInt()] = element - this.size = size + 1 - this.tail = tail + 1 - } - // if offered successfully, then check subscribers outside of lock - checkSubOffers() - return OFFER_SUCCESS - } - - // result is `ALREADY_SELECTED | OFFER_SUCCESS | OFFER_FAILED | Closed` - override fun offerSelectInternal(element: E, select: SelectInstance<*>): Any { - bufferLock.withLock { - // check if closed for send (under lock, so size cannot change) - closedForSend?.let { return it } - val size = this.size - if (size >= capacity) return OFFER_FAILED - // let's try to select sending this element to buffer - if (!select.trySelect()) { // :todo: move trySelect completion outside of lock - return ALREADY_SELECTED - } - val tail = this.tail - buffer[(tail % capacity).toInt()] = element - this.size = size + 1 - this.tail = tail + 1 - } - // if offered successfully, then check subscribers outside of lock - checkSubOffers() - return OFFER_SUCCESS - } - - private fun checkSubOffers() { - var updated = false - var hasSubs = false - @Suppress("LoopToCallChain") // must invoke `checkOffer` on every sub - for (sub in subscribers) { - hasSubs = true - if (sub.checkOffer()) updated = true - } - if (updated || !hasSubs) - updateHead() - } - - // updates head if needed and optionally adds / removes subscriber under the same lock - private tailrec fun updateHead(addSub: Subscriber<E>? = null, removeSub: Subscriber<E>? = null) { - // update head in a tail rec loop - var send: Send? = null - bufferLock.withLock { - if (addSub != null) { - addSub.subHead = tail // start from last element - val wasEmpty = subscribers.isEmpty() - subscribers.add(addSub) - if (!wasEmpty) return // no need to update when adding second and etc sub - } - if (removeSub != null) { - subscribers.remove(removeSub) - if (head != removeSub.subHead) return // no need to update - } - val minHead = computeMinHead() - val tail = this.tail - var head = this.head - val targetHead = minHead.coerceAtMost(tail) - if (targetHead <= head) return // nothing to do -- head was already moved - var size = this.size - // clean up removed (on not need if we don't have any subscribers anymore) - while (head < targetHead) { - buffer[(head % capacity).toInt()] = null - val wasFull = size >= capacity - // update the size before checking queue (no more senders can queue up) - this.head = ++head - this.size = --size - if (wasFull) { - while (true) { - send = takeFirstSendOrPeekClosed() ?: break // when when no sender - if (send is Closed<*>) break // break when closed for send - val token = send!!.tryResumeSend(null) - if (token != null) { - assert { token === RESUME_TOKEN } - // put sent element to the buffer - buffer[(tail % capacity).toInt()] = (send as Send).pollResult - this.size = size + 1 - this.tail = tail + 1 - return@withLock // go out of lock to wakeup this sender - } - // Too late, already cancelled, but we removed it from the queue and need to release resources. - // However, ArrayBroadcastChannel does not support onUndeliveredElement, so nothing to do - } - } - } - return // done updating here -> return - } - // we only get out of the lock normally when there is a sender to resume - send!!.completeResumeSend() - // since we've just sent an element, we might need to resume some receivers - checkSubOffers() - // tailrec call to recheck - updateHead() - } - - private fun computeMinHead(): Long { - var minHead = Long.MAX_VALUE - for (sub in subscribers) - minHead = minHead.coerceAtMost(sub.subHead) // volatile (atomic) reads of subHead - return minHead - } - - @Suppress("UNCHECKED_CAST") - private fun elementAt(index: Long): E = buffer[(index % capacity).toInt()] as E - - private class Subscriber<E>( - private val broadcastChannel: ArrayBroadcastChannel<E> - ) : AbstractChannel<E>(null), ReceiveChannel<E> { - private val subLock = ReentrantLock() - - private val _subHead = atomic(0L) - var subHead: Long // guarded by subLock - get() = _subHead.value - set(value) { _subHead.value = value } - - override val isBufferAlwaysEmpty: Boolean get() = false - override val isBufferEmpty: Boolean get() = subHead >= broadcastChannel.tail - override val isBufferAlwaysFull: Boolean get() = error("Should not be used") - override val isBufferFull: Boolean get() = error("Should not be used") - - override fun close(cause: Throwable?): Boolean { - val wasClosed = super.close(cause) - if (wasClosed) { - broadcastChannel.updateHead(removeSub = this) - subLock.withLock { - subHead = broadcastChannel.tail - } - } - return wasClosed - } - - // returns true if subHead was updated and broadcast channel's head must be checked - // this method is lock-free (it never waits on lock) - @Suppress("UNCHECKED_CAST") - fun checkOffer(): Boolean { - var updated = false - var closed: Closed<*>? = null - loop@ - while (needsToCheckOfferWithoutLock()) { - // just use `tryLock` here and break when some other thread is checking under lock - // it means that `checkOffer` must be retried after every `unlock` - if (!subLock.tryLock()) break - val receive: ReceiveOrClosed<E>? - var result: Any? - try { - result = peekUnderLock() - when { - result === POLL_FAILED -> continue@loop // must retest `needsToCheckOfferWithoutLock` outside of the lock - result is Closed<*> -> { - closed = result - break@loop // was closed - } - } - // find a receiver for an element - receive = takeFirstReceiveOrPeekClosed() ?: break // break when no one's receiving - if (receive is Closed<*>) break // noting more to do if this sub already closed - val token = receive.tryResumeReceive(result as E, null) ?: continue - assert { token === RESUME_TOKEN } - val subHead = this.subHead - this.subHead = subHead + 1 // retrieved element for this subscriber - updated = true - } finally { - subLock.unlock() - } - receive!!.completeResumeReceive(result as E) - } - // do close outside of lock if needed - closed?.also { close(cause = it.closeCause) } - return updated - } - - // result is `E | POLL_FAILED | Closed` - override fun pollInternal(): Any? { - var updated = false - val result = subLock.withLock { - val result = peekUnderLock() - when { - result is Closed<*> -> { /* just bail out of lock */ } - result === POLL_FAILED -> { /* just bail out of lock */ } - else -> { - // update subHead after retrieiving element from buffer - val subHead = this.subHead - this.subHead = subHead + 1 - updated = true - } - } - result - } - // do close outside of lock - (result as? Closed<*>)?.also { close(cause = it.closeCause) } - // there could have been checkOffer attempt while we were holding lock - // now outside the lock recheck if anything else to offer - if (checkOffer()) - updated = true - // and finally update broadcast's channel head if needed - if (updated) - broadcastChannel.updateHead() - return result - } - - // result is `ALREADY_SELECTED | E | POLL_FAILED | Closed` - override fun pollSelectInternal(select: SelectInstance<*>): Any? { - var updated = false - val result = subLock.withLock { - var result = peekUnderLock() - when { - result is Closed<*> -> { /* just bail out of lock */ } - result === POLL_FAILED -> { /* just bail out of lock */ } - else -> { - // let's try to select receiving this element from buffer - if (!select.trySelect()) { // :todo: move trySelect completion outside of lock - result = ALREADY_SELECTED - } else { - // update subHead after retrieiving element from buffer - val subHead = this.subHead - this.subHead = subHead + 1 - updated = true - } - } - } - result - } - // do close outside of lock - (result as? Closed<*>)?.also { close(cause = it.closeCause) } - // there could have been checkOffer attempt while we were holding lock - // now outside the lock recheck if anything else to offer - if (checkOffer()) - updated = true - // and finally update broadcast's channel head if needed - if (updated) - broadcastChannel.updateHead() - return result - } - - // Must invoke this check this after lock, because offer's invocation of `checkOffer` might have failed - // to `tryLock` just before the lock was about to unlocked, thus loosing notification to this - // subscription about an element that was just offered - private fun needsToCheckOfferWithoutLock(): Boolean { - if (closedForReceive != null) - return false // already closed -> nothing to do - if (isBufferEmpty && broadcastChannel.closedForReceive == null) - return false // no data for us && broadcast channel was not closed yet -> nothing to do - return true // check otherwise - } - - // guarded by lock, returns: - // E - the element from the buffer at subHead - // Closed<*> when closed; - // POLL_FAILED when there seems to be no data, but must retest `needsToCheckOfferWithoutLock` outside of lock - private fun peekUnderLock(): Any? { - val subHead = this.subHead // guarded read (can be non-volatile read) - // note: from the broadcastChannel we must read closed token first, then read its tail - // because it is Ok if tail moves in between the reads (we make decision based on tail first) - val closedBroadcast = broadcastChannel.closedForReceive // unguarded volatile read - val tail = broadcastChannel.tail // unguarded volatile read - if (subHead >= tail) { - // no elements to poll from the queue -- check if closed broads & closed this sub - // must retest `needsToCheckOfferWithoutLock` outside of the lock - return closedBroadcast ?: this.closedForReceive ?: POLL_FAILED - } - // Get tentative result. This result may be wrong (completely invalid value, including null), - // because this subscription might get closed, moving channel's head past this subscription's head. - val result = broadcastChannel.elementAt(subHead) - // now check if this subscription was closed - val closedSub = this.closedForReceive - if (closedSub != null) return closedSub - // we know the subscription was not closed, so this tentative result is Ok to return - return result - } - } - - // ------ debug ------ - - override val bufferDebugString: String - get() = "(buffer:capacity=${buffer.size},size=$size)" -} diff --git a/kotlinx-coroutines-core/common/src/channels/ArrayChannel.kt b/kotlinx-coroutines-core/common/src/channels/ArrayChannel.kt deleted file mode 100644 index 7e6c0e68..00000000 --- a/kotlinx-coroutines-core/common/src/channels/ArrayChannel.kt +++ /dev/null @@ -1,307 +0,0 @@ -/* - * Copyright 2016-2021 JetBrains s.r.o. Use of this source code is governed by the Apache 2.0 license. - */ - -package kotlinx.coroutines.channels - -import kotlinx.atomicfu.* -import kotlinx.coroutines.* -import kotlinx.coroutines.internal.* -import kotlinx.coroutines.selects.* -import kotlin.math.* - -/** - * Channel with array buffer of a fixed [capacity]. - * Sender suspends only when buffer is full and receiver suspends only when buffer is empty. - * - * This channel is created by `Channel(capacity)` factory function invocation. - * - * This implementation uses lock to protect the buffer, which is held only during very short buffer-update operations. - * The lists of suspended senders or receivers are lock-free. - **/ -internal open class ArrayChannel<E>( - /** - * Buffer capacity. - */ - private val capacity: Int, - private val onBufferOverflow: BufferOverflow, - onUndeliveredElement: OnUndeliveredElement<E>? -) : AbstractChannel<E>(onUndeliveredElement) { - init { - // This check is actually used by the Channel(...) constructor function which checks only for known - // capacities and calls ArrayChannel constructor for everything else. - require(capacity >= 1) { "ArrayChannel capacity must be at least 1, but $capacity was specified" } - } - - private val lock = ReentrantLock() - - /* - * Guarded by lock. - * Allocate minimum of capacity and 16 to avoid excess memory pressure for large channels when it's not necessary. - */ - private var buffer: Array<Any?> = arrayOfNulls<Any?>(min(capacity, 8)).apply { fill(EMPTY) } - - private var head: Int = 0 - private val size = atomic(0) // Invariant: size <= capacity - - protected final override val isBufferAlwaysEmpty: Boolean get() = false - protected final override val isBufferEmpty: Boolean get() = size.value == 0 - protected final override val isBufferAlwaysFull: Boolean get() = false - protected final override val isBufferFull: Boolean get() = size.value == capacity && onBufferOverflow == BufferOverflow.SUSPEND - - override val isEmpty: Boolean get() = lock.withLock { isEmptyImpl } - override val isClosedForReceive: Boolean get() = lock.withLock { super.isClosedForReceive } - - // result is `OFFER_SUCCESS | OFFER_FAILED | Closed` - protected override fun offerInternal(element: E): Any { - var receive: ReceiveOrClosed<E>? = null - lock.withLock { - val size = this.size.value - closedForSend?.let { return it } - // update size before checking queue (!!!) - updateBufferSize(size)?.let { return it } - // check for receivers that were waiting on empty queue - if (size == 0) { - loop@ while (true) { - receive = takeFirstReceiveOrPeekClosed() ?: break@loop // break when no receivers queued - if (receive is Closed) { - this.size.value = size // restore size - return receive!! - } - val token = receive!!.tryResumeReceive(element, null) - if (token != null) { - assert { token === RESUME_TOKEN } - this.size.value = size // restore size - return@withLock - } - } - } - enqueueElement(size, element) - return OFFER_SUCCESS - } - // breaks here if offer meets receiver - receive!!.completeResumeReceive(element) - return receive!!.offerResult - } - - // result is `ALREADY_SELECTED | OFFER_SUCCESS | OFFER_FAILED | Closed` - protected override fun offerSelectInternal(element: E, select: SelectInstance<*>): Any { - var receive: ReceiveOrClosed<E>? = null - lock.withLock { - val size = this.size.value - closedForSend?.let { return it } - // update size before checking queue (!!!) - updateBufferSize(size)?.let { return it } - // check for receivers that were waiting on empty queue - if (size == 0) { - loop@ while (true) { - val offerOp = describeTryOffer(element) - val failure = select.performAtomicTrySelect(offerOp) - when { - failure == null -> { // offered successfully - this.size.value = size // restore size - receive = offerOp.result - return@withLock - } - failure === OFFER_FAILED -> break@loop // cannot offer -> Ok to queue to buffer - failure === RETRY_ATOMIC -> {} // retry - failure === ALREADY_SELECTED || failure is Closed<*> -> { - this.size.value = size // restore size - return failure - } - else -> error("performAtomicTrySelect(describeTryOffer) returned $failure") - } - } - } - // let's try to select sending this element to buffer - if (!select.trySelect()) { // :todo: move trySelect completion outside of lock - this.size.value = size // restore size - return ALREADY_SELECTED - } - enqueueElement(size, element) - return OFFER_SUCCESS - } - // breaks here if offer meets receiver - receive!!.completeResumeReceive(element) - return receive!!.offerResult - } - - override fun enqueueSend(send: Send): Any? = lock.withLock { - super.enqueueSend(send) - } - - // Guarded by lock - // Result is `OFFER_SUCCESS | OFFER_FAILED | null` - private fun updateBufferSize(currentSize: Int): Symbol? { - if (currentSize < capacity) { - size.value = currentSize + 1 // tentatively put it into the buffer - return null // proceed - } - // buffer is full - return when (onBufferOverflow) { - BufferOverflow.SUSPEND -> OFFER_FAILED - BufferOverflow.DROP_LATEST -> OFFER_SUCCESS - BufferOverflow.DROP_OLDEST -> null // proceed, will drop oldest in enqueueElement - } - } - - // Guarded by lock - private fun enqueueElement(currentSize: Int, element: E) { - if (currentSize < capacity) { - ensureCapacity(currentSize) - buffer[(head + currentSize) % buffer.size] = element // actually queue element - } else { - // buffer is full - assert { onBufferOverflow == BufferOverflow.DROP_OLDEST } // the only way we can get here - buffer[head % buffer.size] = null // drop oldest element - buffer[(head + currentSize) % buffer.size] = element // actually queue element - head = (head + 1) % buffer.size - } - } - - // Guarded by lock - private fun ensureCapacity(currentSize: Int) { - if (currentSize >= buffer.size) { - val newSize = min(buffer.size * 2, capacity) - val newBuffer = arrayOfNulls<Any?>(newSize) - for (i in 0 until currentSize) { - newBuffer[i] = buffer[(head + i) % buffer.size] - } - newBuffer.fill(EMPTY, currentSize, newSize) - buffer = newBuffer - head = 0 - } - } - - // result is `E | POLL_FAILED | Closed` - protected override fun pollInternal(): Any? { - var send: Send? = null - var resumed = false - var result: Any? = null - lock.withLock { - val size = this.size.value - if (size == 0) return closedForSend ?: POLL_FAILED // when nothing can be read from buffer - // size > 0: not empty -- retrieve element - result = buffer[head] - buffer[head] = null - this.size.value = size - 1 // update size before checking queue (!!!) - // check for senders that were waiting on full queue - var replacement: Any? = POLL_FAILED - if (size == capacity) { - loop@ while (true) { - send = takeFirstSendOrPeekClosed() ?: break - val token = send!!.tryResumeSend(null) - if (token != null) { - assert { token === RESUME_TOKEN } - resumed = true - replacement = send!!.pollResult - break@loop - } - // too late, already cancelled, but we removed it from the queue and need to notify on undelivered element - send!!.undeliveredElement() - } - } - if (replacement !== POLL_FAILED && replacement !is Closed<*>) { - this.size.value = size // restore size - buffer[(head + size) % buffer.size] = replacement - } - head = (head + 1) % buffer.size - } - // complete send the we're taken replacement from - if (resumed) - send!!.completeResumeSend() - return result - } - - // result is `ALREADY_SELECTED | E | POLL_FAILED | Closed` - protected override fun pollSelectInternal(select: SelectInstance<*>): Any? { - var send: Send? = null - var success = false - var result: Any? = null - lock.withLock { - val size = this.size.value - if (size == 0) return closedForSend ?: POLL_FAILED - // size > 0: not empty -- retrieve element - result = buffer[head] - buffer[head] = null - this.size.value = size - 1 // update size before checking queue (!!!) - // check for senders that were waiting on full queue - var replacement: Any? = POLL_FAILED - if (size == capacity) { - loop@ while (true) { - val pollOp = describeTryPoll() - val failure = select.performAtomicTrySelect(pollOp) - when { - failure == null -> { // polled successfully - send = pollOp.result - success = true - replacement = send!!.pollResult - break@loop - } - failure === POLL_FAILED -> break@loop // cannot poll -> Ok to take from buffer - failure === RETRY_ATOMIC -> {} // retry - failure === ALREADY_SELECTED -> { - this.size.value = size // restore size - buffer[head] = result // restore head - return failure - } - failure is Closed<*> -> { - send = failure - success = true - replacement = failure - break@loop - } - else -> error("performAtomicTrySelect(describeTryOffer) returned $failure") - } - } - } - if (replacement !== POLL_FAILED && replacement !is Closed<*>) { - this.size.value = size // restore size - buffer[(head + size) % buffer.size] = replacement - } else { - // failed to poll or is already closed --> let's try to select receiving this element from buffer - if (!select.trySelect()) { // :todo: move trySelect completion outside of lock - this.size.value = size // restore size - buffer[head] = result // restore head - return ALREADY_SELECTED - } - } - head = (head + 1) % buffer.size - } - // complete send the we're taken replacement from - if (success) - send!!.completeResumeSend() - return result - } - - override fun enqueueReceiveInternal(receive: Receive<E>): Boolean = lock.withLock { - super.enqueueReceiveInternal(receive) - } - - // Note: this function is invoked when channel is already closed - override fun onCancelIdempotent(wasClosed: Boolean) { - // clear buffer first, but do not wait for it in helpers - val onUndeliveredElement = onUndeliveredElement - var undeliveredElementException: UndeliveredElementException? = null // first cancel exception, others suppressed - lock.withLock { - repeat(size.value) { - val value = buffer[head] - if (onUndeliveredElement != null && value !== EMPTY) { - @Suppress("UNCHECKED_CAST") - undeliveredElementException = onUndeliveredElement.callUndeliveredElementCatchingException(value as E, undeliveredElementException) - } - buffer[head] = EMPTY - head = (head + 1) % buffer.size - } - size.value = 0 - } - // then clean all queued senders - super.onCancelIdempotent(wasClosed) - undeliveredElementException?.let { throw it } // throw UndeliveredElementException at the end if there was one - } - - // ------ debug ------ - - override val bufferDebugString: String - get() = "(buffer:capacity=$capacity,size=${size.value})" -} diff --git a/kotlinx-coroutines-core/common/src/channels/Broadcast.kt b/kotlinx-coroutines-core/common/src/channels/Broadcast.kt index b1c24b45..e7a58ccd 100644 --- a/kotlinx-coroutines-core/common/src/channels/Broadcast.kt +++ b/kotlinx-coroutines-core/common/src/channels/Broadcast.kt @@ -2,6 +2,8 @@ * Copyright 2016-2021 JetBrains s.r.o. Use of this source code is governed by the Apache 2.0 license. */ +@file:Suppress("DEPRECATION") + package kotlinx.coroutines.channels import kotlinx.coroutines.* @@ -16,7 +18,7 @@ import kotlin.coroutines.intrinsics.* * This function [consumes][ReceiveChannel.consume] all elements of the original [ReceiveChannel]. * * The kind of the resulting channel depends on the specified [capacity] parameter: - * when `capacity` is positive (1 by default), but less than [UNLIMITED] -- uses `ArrayBroadcastChannel` with a buffer of given capacity, + * when `capacity` is positive (1 by default), but less than [UNLIMITED] -- uses [BroadcastChannel] with a buffer of given capacity, * when `capacity` is [CONFLATED] -- uses [ConflatedBroadcastChannel] that conflates back-to-back sends; * Note that resulting channel behaves like [ConflatedBroadcastChannel] but is not an instance of [ConflatedBroadcastChannel]. * otherwise -- throws [IllegalArgumentException]. @@ -35,22 +37,23 @@ import kotlin.coroutines.intrinsics.* * [send][BroadcastChannel.send] and [close][BroadcastChannel.close] operations that interfere with * the broadcasting coroutine in hard-to-specify ways. * - * **Note: This API is obsolete since 1.5.0.** It will be deprecated with warning in 1.6.0 - * and with error in 1.7.0. It is replaced with [Flow.shareIn][kotlinx.coroutines.flow.shareIn] - * operator. + * **Note: This API is obsolete since 1.5.0.** It is deprecated with warning in 1.7.0. + * It is replaced with [Flow.shareIn][kotlinx.coroutines.flow.shareIn] operator. * * @param start coroutine start option. The default value is [CoroutineStart.LAZY]. */ @ObsoleteCoroutinesApi +@Deprecated(level = DeprecationLevel.WARNING, message = "BroadcastChannel is deprecated in the favour of SharedFlow and is no longer supported") public fun <E> ReceiveChannel<E>.broadcast( capacity: Int = 1, start: CoroutineStart = CoroutineStart.LAZY ): BroadcastChannel<E> { val scope = GlobalScope + Dispatchers.Unconfined + CoroutineExceptionHandler { _, _ -> } + val channel = this // We can run this coroutine in the context that ignores all exceptions, because of `onCompletion = consume()` // which passes all exceptions upstream to the source ReceiveChannel return scope.broadcast(capacity = capacity, start = start, onCompletion = { cancelConsumed(it) }) { - for (e in this@broadcast) { + for (e in channel) { send(e) } } @@ -75,7 +78,7 @@ public fun <E> ReceiveChannel<E>.broadcast( * the resulting channel becomes _failed_, so that any attempt to receive from such a channel throws exception. * * The kind of the resulting channel depends on the specified [capacity] parameter: - * * when `capacity` is positive (1 by default), but less than [UNLIMITED] -- uses `ArrayBroadcastChannel` with a buffer of given capacity, + * * when `capacity` is positive (1 by default), but less than [UNLIMITED] -- uses [BroadcastChannel] with a buffer of given capacity, * * when `capacity` is [CONFLATED] -- uses [ConflatedBroadcastChannel] that conflates back-to-back sends; * Note that resulting channel behaves like [ConflatedBroadcastChannel] but is not an instance of [ConflatedBroadcastChannel]. * * otherwise -- throws [IllegalArgumentException]. @@ -97,12 +100,11 @@ public fun <E> ReceiveChannel<E>.broadcast( * * ### Future replacement * - * This API is obsolete since 1.5.0. + * This API is obsolete since 1.5.0 and deprecated with warning since 1.7.0. * This function has an inappropriate result type of [BroadcastChannel] which provides * [send][BroadcastChannel.send] and [close][BroadcastChannel.close] operations that interfere with - * the broadcasting coroutine in hard-to-specify ways. It will be deprecated with warning in 1.6.0 - * and with error in 1.7.0. It is replaced with [Flow.shareIn][kotlinx.coroutines.flow.shareIn] - * operator. + * the broadcasting coroutine in hard-to-specify ways. + * It is replaced with [Flow.shareIn][kotlinx.coroutines.flow.shareIn] operator. * * @param context additional to [CoroutineScope.coroutineContext] context of the coroutine. * @param capacity capacity of the channel's buffer (1 by default). @@ -111,6 +113,7 @@ public fun <E> ReceiveChannel<E>.broadcast( * @param block the coroutine code. */ @ObsoleteCoroutinesApi +@Deprecated(level = DeprecationLevel.WARNING, message = "BroadcastChannel is deprecated in the favour of SharedFlow and is no longer supported") public fun <E> CoroutineScope.broadcast( context: CoroutineContext = EmptyCoroutineContext, capacity: Int = 1, diff --git a/kotlinx-coroutines-core/common/src/channels/BroadcastChannel.kt b/kotlinx-coroutines-core/common/src/channels/BroadcastChannel.kt index c82b8dbd..e3c3a306 100644 --- a/kotlinx-coroutines-core/common/src/channels/BroadcastChannel.kt +++ b/kotlinx-coroutines-core/common/src/channels/BroadcastChannel.kt @@ -2,15 +2,19 @@ * Copyright 2016-2021 JetBrains s.r.o. Use of this source code is governed by the Apache 2.0 license. */ -@file:Suppress("FunctionName") +@file:Suppress("FunctionName", "DEPRECATION") package kotlinx.coroutines.channels import kotlinx.coroutines.* +import kotlinx.coroutines.channels.BufferOverflow.* import kotlinx.coroutines.channels.Channel.Factory.BUFFERED import kotlinx.coroutines.channels.Channel.Factory.CHANNEL_DEFAULT_CAPACITY import kotlinx.coroutines.channels.Channel.Factory.CONFLATED import kotlinx.coroutines.channels.Channel.Factory.UNLIMITED +import kotlinx.coroutines.internal.* +import kotlinx.coroutines.selects.* +import kotlin.native.concurrent.* /** * Broadcast channel is a non-blocking primitive for communication between the sender and multiple receivers @@ -20,10 +24,11 @@ import kotlinx.coroutines.channels.Channel.Factory.UNLIMITED * See `BroadcastChannel()` factory function for the description of available * broadcast channel implementations. * - * **Note: This API is obsolete since 1.5.0.** It will be deprecated with warning in 1.6.0 - * and with error in 1.7.0. It is replaced with [SharedFlow][kotlinx.coroutines.flow.SharedFlow]. + * **Note: This API is obsolete since 1.5.0 and deprecated for removal since 1.7.0** + * It is replaced with [SharedFlow][kotlinx.coroutines.flow.SharedFlow]. */ @ObsoleteCoroutinesApi +@Deprecated(level = DeprecationLevel.WARNING, message = "BroadcastChannel is deprecated in the favour of SharedFlow and is no longer supported") public interface BroadcastChannel<E> : SendChannel<E> { /** * Subscribes to this [BroadcastChannel] and returns a channel to receive elements from it. @@ -55,21 +60,354 @@ public interface BroadcastChannel<E> : SendChannel<E> { * The resulting channel type depends on the specified [capacity] parameter: * * * when `capacity` positive, but less than [UNLIMITED] -- creates `ArrayBroadcastChannel` with a buffer of given capacity. - * **Note:** this channel looses all items that are send to it until the first subscriber appears; + * **Note:** this channel looses all items that have been sent to it until the first subscriber appears; * * when `capacity` is [CONFLATED] -- creates [ConflatedBroadcastChannel] that conflates back-to-back sends; * * when `capacity` is [BUFFERED] -- creates `ArrayBroadcastChannel` with a default capacity. * * otherwise -- throws [IllegalArgumentException]. * - * **Note: This API is obsolete since 1.5.0.** It will be deprecated with warning in 1.6.0 - * and with error in 1.7.0. It is replaced with [StateFlow][kotlinx.coroutines.flow.StateFlow] - * and [SharedFlow][kotlinx.coroutines.flow.SharedFlow]. + * **Note: This API is obsolete since 1.5.0 and deprecated for removal since 1.7.0** + * It is replaced with [SharedFlow][kotlinx.coroutines.flow.SharedFlow] and [StateFlow][kotlinx.coroutines.flow.StateFlow]. */ @ObsoleteCoroutinesApi +@Deprecated(level = DeprecationLevel.WARNING, message = "BroadcastChannel is deprecated in the favour of SharedFlow and StateFlow, and is no longer supported") public fun <E> BroadcastChannel(capacity: Int): BroadcastChannel<E> = when (capacity) { 0 -> throw IllegalArgumentException("Unsupported 0 capacity for BroadcastChannel") UNLIMITED -> throw IllegalArgumentException("Unsupported UNLIMITED capacity for BroadcastChannel") CONFLATED -> ConflatedBroadcastChannel() - BUFFERED -> ArrayBroadcastChannel(CHANNEL_DEFAULT_CAPACITY) - else -> ArrayBroadcastChannel(capacity) + BUFFERED -> BroadcastChannelImpl(CHANNEL_DEFAULT_CAPACITY) + else -> BroadcastChannelImpl(capacity) } + +/** + * Broadcasts the most recently sent element (aka [value]) to all [openSubscription] subscribers. + * + * Back-to-send sent elements are _conflated_ -- only the most recently sent value is received, + * while previously sent elements **are lost**. + * Every subscriber immediately receives the most recently sent element. + * Sender to this broadcast channel never suspends and [trySend] always succeeds. + * + * A secondary constructor can be used to create an instance of this class that already holds a value. + * This channel is also created by `BroadcastChannel(Channel.CONFLATED)` factory function invocation. + * + * In this implementation, [opening][openSubscription] and [closing][ReceiveChannel.cancel] subscription + * takes linear time in the number of subscribers. + * + * **Note: This API is obsolete since 1.5.0 and deprecated for removal since 1.7.0** + * It is replaced with [SharedFlow][kotlinx.coroutines.flow.StateFlow]. + */ +@ObsoleteCoroutinesApi +@Deprecated(level = DeprecationLevel.WARNING, message = "ConflatedBroadcastChannel is deprecated in the favour of SharedFlow and is no longer supported") +public class ConflatedBroadcastChannel<E> private constructor( + private val broadcast: BroadcastChannelImpl<E> +) : BroadcastChannel<E> by broadcast { + public constructor(): this(BroadcastChannelImpl<E>(capacity = CONFLATED)) + /** + * Creates an instance of this class that already holds a value. + * + * It is as a shortcut to creating an instance with a default constructor and + * immediately sending an element: `ConflatedBroadcastChannel().apply { offer(value) }`. + */ + public constructor(value: E) : this() { + trySend(value) + } + + /** + * The most recently sent element to this channel. + * + * Access to this property throws [IllegalStateException] when this class is constructed without + * initial value and no value was sent yet or if it was [closed][close] without a cause. + * It throws the original [close][SendChannel.close] cause exception if the channel has _failed_. + */ + public val value: E get() = broadcast.value + /** + * The most recently sent element to this channel or `null` when this class is constructed without + * initial value and no value was sent yet or if it was [closed][close]. + */ + public val valueOrNull: E? get() = broadcast.valueOrNull +} + +/** + * A common implementation for both the broadcast channel with a buffer of fixed [capacity] + * and the conflated broadcast channel (see [ConflatedBroadcastChannel]). + * + * **Note**, that elements that are sent to this channel while there are no + * [openSubscription] subscribers are immediately lost. + * + * This channel is created by `BroadcastChannel(capacity)` factory function invocation. + */ +internal class BroadcastChannelImpl<E>( + /** + * Buffer capacity; [Channel.CONFLATED] when this broadcast is conflated. + */ + val capacity: Int +) : BufferedChannel<E>(capacity = Channel.RENDEZVOUS, onUndeliveredElement = null), BroadcastChannel<E> { + init { + require(capacity >= 1 || capacity == CONFLATED) { + "BroadcastChannel capacity must be positive or Channel.CONFLATED, but $capacity was specified" + } + } + + // This implementation uses coarse-grained synchronization, + // as, reputedly, it is the simplest synchronization scheme. + // All operations are protected by this lock. + private val lock = ReentrantLock() + // The list of subscribers; all accesses should be protected by lock. + // Each change must create a new list instance to avoid `ConcurrentModificationException`. + private var subscribers: List<BufferedChannel<E>> = emptyList() + // When this broadcast is conflated, this field stores the last sent element. + // If this channel is empty or not conflated, it stores a special `NO_ELEMENT` marker. + private var lastConflatedElement: Any? = NO_ELEMENT // NO_ELEMENT or E + + // ########################### + // # Subscription Management # + // ########################### + + override fun openSubscription(): ReceiveChannel<E> = lock.withLock { // protected by lock + // Is this broadcast conflated or buffered? + // Create the corresponding subscription channel. + val s = if (capacity == CONFLATED) SubscriberConflated() else SubscriberBuffered() + // If this broadcast is already closed or cancelled, + // and the last sent element is not available in case + // this broadcast is conflated, close the created + // subscriber immediately and return it. + if (isClosedForSend && lastConflatedElement === NO_ELEMENT) { + s.close(closeCause) + return s + } + // Is this broadcast conflated? If so, send + // the last sent element to the subscriber. + if (lastConflatedElement !== NO_ELEMENT) { + s.trySend(value) + } + // Add the subscriber to the list and return it. + subscribers += s + s + } + + private fun removeSubscriber(s: ReceiveChannel<E>) = lock.withLock { // protected by lock + subscribers = subscribers.filter { it !== s } + } + + // ############################# + // # The `send(..)` Operations # + // ############################# + + /** + * Sends the specified element to all subscribers. + * + * **!!! THIS IMPLEMENTATION IS NOT LINEARIZABLE !!!** + * + * As the operation should send the element to multiple + * subscribers simultaneously, it is non-trivial to + * implement it in an atomic way. Specifically, this + * would require a special implementation that does + * not transfer the element until all parties are able + * to resume it (this `send(..)` can be cancelled + * or the broadcast can become closed in the meantime). + * As broadcasts are obsolete, we keep this implementation + * as simple as possible, allowing non-linearizability + * in corner cases. + */ + override suspend fun send(element: E) { + val subs = lock.withLock { // protected by lock + // Is this channel closed for send? + if (isClosedForSend) throw sendException + // Update the last sent element if this broadcast is conflated. + if (capacity == CONFLATED) lastConflatedElement = element + // Get a reference to the list of subscribers under the lock. + subscribers + } + // The lock has been released. Send the element to the + // subscribers one-by-one, and finish immediately + // when this broadcast discovered in the closed state. + // Note that this implementation is non-linearizable; + // see this method documentation for details. + subs.forEach { + // We use special function to send the element, + // which returns `true` on success and `false` + // if the subscriber is closed. + val success = it.sendBroadcast(element) + // The sending attempt has failed. + // Check whether the broadcast is closed. + if (!success && isClosedForSend) throw sendException + } + } + + override fun trySend(element: E): ChannelResult<Unit> = lock.withLock { // protected by lock + // Is this channel closed for send? + if (isClosedForSend) return super.trySend(element) + // Check whether the plain `send(..)` operation + // should suspend and fail in this case. + val shouldSuspend = subscribers.any { it.shouldSendSuspend() } + if (shouldSuspend) return ChannelResult.failure() + // Update the last sent element if this broadcast is conflated. + if (capacity == CONFLATED) lastConflatedElement = element + // Send the element to all subscribers. + // It is guaranteed that the attempt cannot fail, + // as both the broadcast closing and subscription + // cancellation are guarded by lock, which is held + // by the current operation. + subscribers.forEach { it.trySend(element) } + // Finish with success. + return ChannelResult.success(Unit) + } + + // ########################################### + // # The `select` Expression: onSend { ... } # + // ########################################### + + override fun registerSelectForSend(select: SelectInstance<*>, element: Any?) { + // It is extremely complicated to support sending via `select` for broadcasts, + // as the operation should wait on multiple subscribers simultaneously. + // At the same time, broadcasts are obsolete, so we need a simple implementation + // that works somehow. Here is a tricky work-around. First, we launch a new + // coroutine that performs plain `send(..)` operation and tries to complete + // this `select` via `trySelect`, independently on whether it is in the + // registration or in the waiting phase. On success, the operation finishes. + // On failure, if another clause is already selected or the `select` operation + // has been cancelled, we observe non-linearizable behaviour, as this `onSend` + // clause is completed as well. However, we believe that such a non-linearizability + // is fine for obsolete API. The last case is when the `select` operation is still + // in the registration case, so this `onSend` clause should be re-registered. + // The idea is that we keep information that this `onSend` clause is already selected + // and finish immediately. + @Suppress("UNCHECKED_CAST") + element as E + // First, check whether this `onSend` clause is already + // selected, finishing immediately in this case. + lock.withLock { + val result = onSendInternalResult.remove(select) + if (result != null) { // already selected! + // `result` is either `Unit` ot `CHANNEL_CLOSED`. + select.selectInRegistrationPhase(result) + return + } + } + // Start a new coroutine that performs plain `send(..)` + // and tries to select this `onSend` clause at the end. + CoroutineScope(select.context).launch(start = CoroutineStart.UNDISPATCHED) { + val success: Boolean = try { + send(element) + // The element has been successfully sent! + true + } catch (t: Throwable) { + // This broadcast must be closed. However, it is possible that + // an unrelated exception, such as `OutOfMemoryError` has been thrown. + // This implementation checks that the channel is actually closed, + // re-throwing the caught exception otherwise. + if (isClosedForSend && (t is ClosedSendChannelException || sendException === t)) false + else throw t + } + // Mark this `onSend` clause as selected and + // try to complete the `select` operation. + lock.withLock { + // Status of this `onSend` clause should not be presented yet. + assert { onSendInternalResult[select] == null } + // Success or fail? Put the corresponding result. + onSendInternalResult[select] = if (success) Unit else CHANNEL_CLOSED + // Try to select this `onSend` clause. + select as SelectImplementation<*> + val trySelectResult = select.trySelectDetailed(this@BroadcastChannelImpl, Unit) + if (trySelectResult !== TrySelectDetailedResult.REREGISTER) { + // In case of re-registration (this `select` was still + // in the registration phase), the algorithm will invoke + // `registerSelectForSend`. As we stored an information that + // this `onSend` clause is already selected (in `onSendInternalResult`), + // the algorithm, will complete immediately. Otherwise, to avoid memory + // leaks, we must remove this information from the hashmap. + onSendInternalResult.remove(select) + } + } + + } + } + private val onSendInternalResult = HashMap<SelectInstance<*>, Any?>() // select -> Unit or CHANNEL_CLOSED + + // ############################ + // # Closing and Cancellation # + // ############################ + + override fun close(cause: Throwable?): Boolean = lock.withLock { // protected by lock + // Close all subscriptions first. + subscribers.forEach { it.close(cause) } + // Remove all subscriptions that do not contain + // buffered elements or waiting send-s to avoid + // memory leaks. We must keep other subscriptions + // in case `broadcast.cancel(..)` is called. + subscribers = subscribers.filter { it.hasElements() } + // Delegate to the parent implementation. + super.close(cause) + } + + override fun cancelImpl(cause: Throwable?): Boolean = lock.withLock { // protected by lock + // Cancel all subscriptions. As part of cancellation procedure, + // subscriptions automatically remove themselves from this broadcast. + subscribers.forEach { it.cancelImpl(cause) } + // For the conflated implementation, clear the last sent element. + lastConflatedElement = NO_ELEMENT + // Finally, delegate to the parent implementation. + super.cancelImpl(cause) + } + + override val isClosedForSend: Boolean + // Protect by lock to synchronize with `close(..)` / `cancel(..)`. + get() = lock.withLock { super.isClosedForSend } + + // ############################## + // # Subscriber Implementations # + // ############################## + + private inner class SubscriberBuffered : BufferedChannel<E>(capacity = capacity) { + public override fun cancelImpl(cause: Throwable?): Boolean = lock.withLock { + // Remove this subscriber from the broadcast on cancellation. + removeSubscriber(this@SubscriberBuffered ) + super.cancelImpl(cause) + } + } + + private inner class SubscriberConflated : ConflatedBufferedChannel<E>(capacity = 1, onBufferOverflow = DROP_OLDEST) { + public override fun cancelImpl(cause: Throwable?): Boolean { + // Remove this subscriber from the broadcast on cancellation. + removeSubscriber(this@SubscriberConflated ) + return super.cancelImpl(cause) + } + } + + // ######################################## + // # ConflatedBroadcastChannel Operations # + // ######################################## + + @Suppress("UNCHECKED_CAST") + val value: E get() = lock.withLock { + // Is this channel closed for sending? + if (isClosedForSend) { + throw closeCause ?: IllegalStateException("This broadcast channel is closed") + } + // Is there sent element? + if (lastConflatedElement === NO_ELEMENT) error("No value") + // Return the last sent element. + lastConflatedElement as E + } + + @Suppress("UNCHECKED_CAST") + val valueOrNull: E? get() = lock.withLock { + // Is this channel closed for sending? + if (isClosedForReceive) null + // Is there sent element? + else if (lastConflatedElement === NO_ELEMENT) null + // Return the last sent element. + else lastConflatedElement as E + } + + // ################# + // # For Debugging # + // ################# + + override fun toString() = + (if (lastConflatedElement !== NO_ELEMENT) "CONFLATED_ELEMENT=$lastConflatedElement; " else "") + + "BROADCAST=<${super.toString()}>; " + + "SUBSCRIBERS=${subscribers.joinToString(separator = ";", prefix = "<", postfix = ">")}" +} + +private val NO_ELEMENT = Symbol("NO_ELEMENT") diff --git a/kotlinx-coroutines-core/common/src/channels/BufferOverflow.kt b/kotlinx-coroutines-core/common/src/channels/BufferOverflow.kt index 48b89ce6..8af2c845 100644 --- a/kotlinx-coroutines-core/common/src/channels/BufferOverflow.kt +++ b/kotlinx-coroutines-core/common/src/channels/BufferOverflow.kt @@ -4,8 +4,6 @@ package kotlinx.coroutines.channels -import kotlinx.coroutines.* - /** * A strategy for buffer overflow handling in [channels][Channel] and [flows][kotlinx.coroutines.flow.Flow] that * controls what is going to be sacrificed on buffer overflow: diff --git a/kotlinx-coroutines-core/common/src/channels/BufferedChannel.kt b/kotlinx-coroutines-core/common/src/channels/BufferedChannel.kt new file mode 100644 index 00000000..4fc7d438 --- /dev/null +++ b/kotlinx-coroutines-core/common/src/channels/BufferedChannel.kt @@ -0,0 +1,3054 @@ +@file:Suppress("PrivatePropertyName") + +package kotlinx.coroutines.channels + +import kotlinx.atomicfu.* +import kotlinx.coroutines.* +import kotlinx.coroutines.channels.ChannelResult.Companion.closed +import kotlinx.coroutines.channels.ChannelResult.Companion.failure +import kotlinx.coroutines.channels.ChannelResult.Companion.success +import kotlinx.coroutines.flow.internal.* +import kotlinx.coroutines.internal.* +import kotlinx.coroutines.selects.* +import kotlinx.coroutines.selects.TrySelectDetailedResult.* +import kotlin.contracts.* +import kotlin.coroutines.* +import kotlin.js.* +import kotlin.jvm.* +import kotlin.math.* +import kotlin.random.* +import kotlin.reflect.* + +/** + * The buffered channel implementation, which also serves as a rendezvous channel when the capacity is zero. + * The high-level structure bases on a conceptually infinite array for storing elements and waiting requests, + * separate counters of [send] and [receive] invocations that were ever performed, and an additional counter + * that indicates the end of the logical buffer by counting the number of array cells it ever contained. + * The key idea is that both [send] and [receive] start by incrementing their counters, assigning the array cell + * referenced by the counter. In case of rendezvous channels, the operation either suspends and stores its continuation + * in the cell or makes a rendezvous with the opposite request. Each cell can be processed by exactly one [send] and + * one [receive]. As for buffered channels, [send]-s can also add elements without suspension if the logical buffer + * contains the cell, while the [receive] operation updates the end of the buffer when its synchronization finishes. + * + * Please see the ["Fast and Scalable Channels in Kotlin Coroutines"](https://arxiv.org/abs/2211.04986) + * paper by Nikita Koval, Roman Elizarov, and Dan Alistarh for the detailed algorithm description. + */ +internal open class BufferedChannel<E>( + /** + * Channel capacity; `Channel.RENDEZVOUS` for rendezvous channel + * and `Channel.UNLIMITED` for unlimited capacity. + */ + private val capacity: Int, + @JvmField + internal val onUndeliveredElement: OnUndeliveredElement<E>? = null +) : Channel<E> { + init { + require(capacity >= 0) { "Invalid channel capacity: $capacity, should be >=0" } + // This implementation has second `init`. + } + + // Maintenance note: use `Buffered1ChannelLincheckTest` to check hypotheses. + + /* + The counters indicate the total numbers of send, receive, and buffer expansion calls + ever performed. The counters are incremented in the beginning of the corresponding + operation; thus, acquiring a unique (for the operation type) cell to process. + The segments reference to the last working one for each operation type. + + Notably, the counter for send is combined with the channel closing status + for synchronization simplicity and performance reasons. + + The logical end of the buffer is initialized with the channel capacity. + If the channel is rendezvous or unlimited, the counter equals `BUFFER_END_RENDEZVOUS` + or `BUFFER_END_RENDEZVOUS`, respectively, and never updates. The `bufferEndSegment` + point to a special `NULL_SEGMENT` in this case. + */ + private val sendersAndCloseStatus = atomic(0L) + private val receivers = atomic(0L) + private val bufferEnd = atomic(initialBufferEnd(capacity)) + + internal val sendersCounter: Long get() = sendersAndCloseStatus.value.sendersCounter + internal val receiversCounter: Long get() = receivers.value + private val bufferEndCounter: Long get() = bufferEnd.value + + /* + Additionally to the counters above, we need an extra one that + tracks the number of cells processed by `expandBuffer()`. + When a receiver aborts, the corresponding cell might be + physically removed from the data structure to avoid memory + leaks, while it still can be unprocessed by `expandBuffer()`. + In this case, `expandBuffer()` cannot know whether the + removed cell contained sender or receiver and, therefore, + cannot proceed. To solve the race, we ensure that cells + correspond to cancelled receivers cannot be physically + removed until the cell is processed. + This additional counter enables the synchronization, + */ + private val completedExpandBuffersAndPauseFlag = atomic(bufferEndCounter) + + private val isRendezvousOrUnlimited + get() = bufferEndCounter.let { it == BUFFER_END_RENDEZVOUS || it == BUFFER_END_UNLIMITED } + + private val sendSegment: AtomicRef<ChannelSegment<E>> + private val receiveSegment: AtomicRef<ChannelSegment<E>> + private val bufferEndSegment: AtomicRef<ChannelSegment<E>> + + init { + @Suppress("LeakingThis") + val firstSegment = ChannelSegment(id = 0, prev = null, channel = this, pointers = 3) + sendSegment = atomic(firstSegment) + receiveSegment = atomic(firstSegment) + // If this channel is rendezvous or has unlimited capacity, the algorithm never + // invokes the buffer expansion procedure, and the corresponding segment reference + // points to a special `NULL_SEGMENT` one and never updates. + @Suppress("UNCHECKED_CAST") + bufferEndSegment = atomic(if (isRendezvousOrUnlimited) (NULL_SEGMENT as ChannelSegment<E>) else firstSegment) + } + + // ######################### + // ## The send operations ## + // ######################### + + override suspend fun send(element: E): Unit = + sendImpl( // <-- this is an inline function + element = element, + // Do not create a continuation until it is required; + // it is created later via [onNoWaiterSuspend], if needed. + waiter = null, + // Finish immediately if a rendezvous happens + // or the element has been buffered. + onRendezvousOrBuffered = {}, + // As no waiter is provided, suspension is impossible. + onSuspend = { _, _ -> assert { false } }, + // According to the `send(e)` contract, we need to call + // `onUndeliveredElement(..)` handler and throw an exception + // if the channel is already closed. + onClosed = { onClosedSend(element) }, + // When `send(e)` decides to suspend, the corresponding + // `onNoWaiterSuspend` function that creates a continuation + // is called. The tail-call optimization is applied here. + onNoWaiterSuspend = { segm, i, elem, s -> sendOnNoWaiterSuspend(segm, i, elem, s) } + ) + + // NB: return type could've been Nothing, but it breaks TCO + private suspend fun onClosedSend(element: E): Unit = suspendCancellableCoroutine { continuation -> + onUndeliveredElement?.callUndeliveredElementCatchingException(element)?.let { + // If it crashes, add send exception as suppressed for better diagnostics + it.addSuppressed(sendException) + continuation.resumeWithStackTrace(it) + return@suspendCancellableCoroutine + } + continuation.resumeWithStackTrace(sendException) + } + + private suspend fun sendOnNoWaiterSuspend( + /* The working cell is specified by + the segment and the index in it. */ + segment: ChannelSegment<E>, + index: Int, + /** The element to be inserted. */ + element: E, + /** The global index of the cell. */ + s: Long + ) = suspendCancellableCoroutineReusable sc@{ cont -> + sendImplOnNoWaiter( // <-- this is an inline function + segment = segment, index = index, element = element, s = s, + // Store the created continuation as a waiter. + waiter = cont, + // If a rendezvous happens or the element has been buffered, + // resume the continuation and finish. In case of prompt + // cancellation, it is guaranteed that the element + // has been already buffered or passed to receiver. + onRendezvousOrBuffered = { cont.resume(Unit) }, + // If the channel is closed, call `onUndeliveredElement(..)` and complete the + // continuation with the corresponding exception. + onClosed = { onClosedSendOnNoWaiterSuspend(element, cont) }, + ) + } + + private fun Waiter.prepareSenderForSuspension( + /* The working cell is specified by + the segment and the index in it. */ + segment: ChannelSegment<E>, + index: Int + ) { + // To distinguish cancelled senders and receivers, + // senders equip the index value with an additional marker, + // adding `SEGMENT_SIZE` to the value. + invokeOnCancellation(segment, index + SEGMENT_SIZE) + } + + private fun onClosedSendOnNoWaiterSuspend(element: E, cont: CancellableContinuation<Unit>) { + onUndeliveredElement?.callUndeliveredElement(element, cont.context) + cont.resumeWithException(recoverStackTrace(sendException, cont)) + } + + override fun trySend(element: E): ChannelResult<Unit> { + // Do not try to send the element if the plain `send(e)` operation would suspend. + if (shouldSendSuspend(sendersAndCloseStatus.value)) return failure() + // This channel either has waiting receivers or is closed. + // Let's try to send the element! + // The logic is similar to the plain `send(e)` operation, with + // the only difference that we install `INTERRUPTED_SEND` in case + // the operation decides to suspend. + return sendImpl( // <-- this is an inline function + element = element, + // Store an already interrupted sender in case of suspension. + waiter = INTERRUPTED_SEND, + // Finish successfully when a rendezvous happens + // or the element has been buffered. + onRendezvousOrBuffered = { success(Unit) }, + // On suspension, the `INTERRUPTED_SEND` token has been installed, + // and this `trySend(e)` must fail. According to the contract, + // we do not need to call the [onUndeliveredElement] handler. + onSuspend = { segm, _ -> + segm.onSlotCleaned() + failure() + }, + // If the channel is closed, return the corresponding result. + onClosed = { closed(sendException) } + ) + } + + /** + * This is a special `send(e)` implementation that returns `true` if the element + * has been successfully sent, and `false` if the channel is closed. + * + * In case of coroutine cancellation, the element may be undelivered -- + * the [onUndeliveredElement] feature is unsupported in this implementation. + * + */ + internal open suspend fun sendBroadcast(element: E): Boolean = suspendCancellableCoroutine { cont -> + check(onUndeliveredElement == null) { + "the `onUndeliveredElement` feature is unsupported for `sendBroadcast(e)`" + } + sendImpl( + element = element, + waiter = SendBroadcast(cont), + onRendezvousOrBuffered = { cont.resume(true) }, + onSuspend = { _, _ -> }, + onClosed = { cont.resume(false) } + ) + } + + /** + * Specifies waiting [sendBroadcast] operation. + */ + private class SendBroadcast( + val cont: CancellableContinuation<Boolean> + ) : Waiter by cont as CancellableContinuationImpl<Boolean> + + /** + * Abstract send implementation. + */ + protected inline fun <R> sendImpl( + /* The element to be sent. */ + element: E, + /* The waiter to be stored in case of suspension, + or `null` if the waiter is not created yet. + In the latter case, when the algorithm decides + to suspend, [onNoWaiterSuspend] is called. */ + waiter: Any?, + /* This lambda is invoked when the element has been + buffered or a rendezvous with a receiver happens. */ + onRendezvousOrBuffered: () -> R, + /* This lambda is called when the operation suspends in the + cell specified by the segment and the index in it. */ + onSuspend: (segm: ChannelSegment<E>, i: Int) -> R, + /* This lambda is called when the channel + is observed in the closed state. */ + onClosed: () -> R, + /* This lambda is called when the operation decides + to suspend, but the waiter is not provided (equals `null`). + It should create a waiter and delegate to `sendImplOnNoWaiter`. */ + onNoWaiterSuspend: ( + segm: ChannelSegment<E>, + i: Int, + element: E, + s: Long + ) -> R = { _, _, _, _ -> error("unexpected") } + ): R { + // Read the segment reference before the counter increment; + // it is crucial to be able to find the required segment later. + var segment = sendSegment.value + while (true) { + // Atomically increment the `senders` counter and obtain the + // value right before the increment along with the close status. + val sendersAndCloseStatusCur = sendersAndCloseStatus.getAndIncrement() + val s = sendersAndCloseStatusCur.sendersCounter + // Is this channel already closed? Keep the information. + val closed = sendersAndCloseStatusCur.isClosedForSend0 + // Count the required segment id and the cell index in it. + val id = s / SEGMENT_SIZE + val i = (s % SEGMENT_SIZE).toInt() + // Try to find the required segment if the initially obtained + // one (in the beginning of this function) has lower id. + if (segment.id != id) { + // Find the required segment. + segment = findSegmentSend(id, segment) ?: + // The required segment has not been found. + // Finish immediately if this channel is closed, + // restarting the operation otherwise. + // In the latter case, the required segment was full + // of interrupted waiters and, therefore, removed + // physically to avoid memory leaks. + if (closed) { + return onClosed() + } else { + continue + } + } + // Update the cell according to the algorithm. Importantly, when + // the channel is already closed, storing a waiter is illegal, so + // the algorithm stores the `INTERRUPTED_SEND` token in this case. + when (updateCellSend(segment, i, element, s, waiter, closed)) { + RESULT_RENDEZVOUS -> { + // A rendezvous with a receiver has happened. + // The previous segments are no longer needed + // for the upcoming requests, so the algorithm + // resets the link to the previous segment. + segment.cleanPrev() + return onRendezvousOrBuffered() + } + RESULT_BUFFERED -> { + // The element has been buffered. + return onRendezvousOrBuffered() + } + RESULT_SUSPEND -> { + // The operation has decided to suspend and installed the + // specified waiter. If the channel was already closed, + // and the `INTERRUPTED_SEND` token has been installed as a waiter, + // this request finishes with the `onClosed()` action. + if (closed) { + segment.onSlotCleaned() + return onClosed() + } + (waiter as? Waiter)?.prepareSenderForSuspension(segment, i) + return onSuspend(segment, i) + } + RESULT_CLOSED -> { + // This channel is closed. + // In case this segment is already or going to be + // processed by a receiver, ensure that all the + // previous segments are unreachable. + if (s < receiversCounter) segment.cleanPrev() + return onClosed() + } + RESULT_FAILED -> { + // Either the cell stores an interrupted receiver, + // or it was poisoned by a concurrent receiver. + // In both cases, all the previous segments are already processed, + segment.cleanPrev() + continue + } + RESULT_SUSPEND_NO_WAITER -> { + // The operation has decided to suspend, + // but no waiter has been provided. + return onNoWaiterSuspend(segment, i, element, s) + } + } + } + } + + private inline fun sendImplOnNoWaiter( + /* The working cell is specified by + the segment and the index in it. */ + segment: ChannelSegment<E>, + index: Int, + /* The element to be sent. */ + element: E, + /* The global index of the cell. */ + s: Long, + /* The waiter to be stored in case of suspension. */ + waiter: Waiter, + /* This lambda is invoked when the element has been + buffered or a rendezvous with a receiver happens.*/ + onRendezvousOrBuffered: () -> Unit, + /* This lambda is called when the channel + is observed in the closed state. */ + onClosed: () -> Unit, + ) { + // Update the cell again, now with the non-null waiter, + // restarting the operation from the beginning on failure. + // Check the `sendImpl(..)` function for the comments. + when (updateCellSend(segment, index, element, s, waiter, false)) { + RESULT_RENDEZVOUS -> { + segment.cleanPrev() + onRendezvousOrBuffered() + } + RESULT_BUFFERED -> { + onRendezvousOrBuffered() + } + RESULT_SUSPEND -> { + waiter.prepareSenderForSuspension(segment, index) + } + RESULT_CLOSED -> { + if (s < receiversCounter) segment.cleanPrev() + onClosed() + } + RESULT_FAILED -> { + segment.cleanPrev() + sendImpl( + element = element, + waiter = waiter, + onRendezvousOrBuffered = onRendezvousOrBuffered, + onSuspend = { _, _ -> }, + onClosed = onClosed, + ) + } + else -> error("unexpected") + } + } + + private fun updateCellSend( + /* The working cell is specified by + the segment and the index in it. */ + segment: ChannelSegment<E>, + index: Int, + /* The element to be sent. */ + element: E, + /* The global index of the cell. */ + s: Long, + /* The waiter to be stored in case of suspension. */ + waiter: Any?, + closed: Boolean + ): Int { + // This is a fast-path of `updateCellSendSlow(..)`. + // + // First, the algorithm stores the element, + // performing the synchronization after that. + // This way, receivers safely retrieve the + // element, following the safe publication pattern. + segment.storeElement(index, element) + if (closed) return updateCellSendSlow(segment, index, element, s, waiter, closed) + // Read the current cell state. + val state = segment.getState(index) + when { + // The cell is empty. + state === null -> { + // If the element should be buffered, or a rendezvous should happen + // while the receiver is still coming, try to buffer the element. + // Otherwise, try to store the specified waiter in the cell. + if (bufferOrRendezvousSend(s)) { + // Move the cell state to `BUFFERED`. + if (segment.casState(index, null, BUFFERED)) { + // The element has been successfully buffered, finish. + return RESULT_BUFFERED + } + } else { + // This `send(e)` operation should suspend. + // However, in case the channel has already + // been observed closed, `INTERRUPTED_SEND` + // is installed instead. + if (waiter == null) { + // The waiter is not specified; return the corresponding result. + return RESULT_SUSPEND_NO_WAITER + } else { + // Try to install the waiter. + if (segment.casState(index, null, waiter)) return RESULT_SUSPEND + } + } + } + // A waiting receiver is stored in the cell. + state is Waiter -> { + // As the element will be passed directly to the waiter, + // the algorithm cleans the element slot in the cell. + segment.cleanElement(index) + // Try to make a rendezvous with the suspended receiver. + return if (state.tryResumeReceiver(element)) { + // Rendezvous! Move the cell state to `DONE_RCV` and finish. + segment.setState(index, DONE_RCV) + onReceiveDequeued() + RESULT_RENDEZVOUS + } else { + // The resumption has failed. Update the cell state correspondingly + // and clean the element field. It is also possible for a concurrent + // cancellation handler to update the cell state; we can safely + // ignore these updates. + if (segment.getAndSetState(index, INTERRUPTED_RCV) !== INTERRUPTED_RCV) { + segment.onCancelledRequest(index, true) + } + RESULT_FAILED + } + } + } + return updateCellSendSlow(segment, index, element, s, waiter, closed) + } + + /** + * Updates the working cell of an abstract send operation. + */ + private fun updateCellSendSlow( + /* The working cell is specified by + the segment and the index in it. */ + segment: ChannelSegment<E>, + index: Int, + /* The element to be sent. */ + element: E, + /* The global index of the cell. */ + s: Long, + /* The waiter to be stored in case of suspension. */ + waiter: Any?, + closed: Boolean + ): Int { + // Then, the cell state should be updated according to + // its state machine; see the paper mentioned in the very + // beginning for the cell life-cycle and the algorithm details. + while (true) { + // Read the current cell state. + val state = segment.getState(index) + when { + // The cell is empty. + state === null -> { + // If the element should be buffered, or a rendezvous should happen + // while the receiver is still coming, try to buffer the element. + // Otherwise, try to store the specified waiter in the cell. + if (bufferOrRendezvousSend(s) && !closed) { + // Move the cell state to `BUFFERED`. + if (segment.casState(index, null, BUFFERED)) { + // The element has been successfully buffered, finish. + return RESULT_BUFFERED + } + } else { + // This `send(e)` operation should suspend. + // However, in case the channel has already + // been observed closed, `INTERRUPTED_SEND` + // is installed instead. + when { + // The channel is closed + closed -> if (segment.casState(index, null, INTERRUPTED_SEND)) { + segment.onCancelledRequest(index, false) + return RESULT_CLOSED + } + // The waiter is not specified; return the corresponding result. + waiter == null -> return RESULT_SUSPEND_NO_WAITER + // Try to install the waiter. + else -> if (segment.casState(index, null, waiter)) return RESULT_SUSPEND + } + } + } + // This cell is in the logical buffer. + state === IN_BUFFER -> { + // Try to buffer the element. + if (segment.casState(index, state, BUFFERED)) { + // The element has been successfully buffered, finish. + return RESULT_BUFFERED + } + } + // The cell stores a cancelled receiver. + state === INTERRUPTED_RCV -> { + // Clean the element slot to avoid memory leaks and finish. + segment.cleanElement(index) + return RESULT_FAILED + } + // The cell is poisoned by a concurrent receive. + state === POISONED -> { + // Clean the element slot to avoid memory leaks and finish. + segment.cleanElement(index) + return RESULT_FAILED + } + // The channel is already closed. + state === CHANNEL_CLOSED -> { + // Clean the element slot to avoid memory leaks, + // ensure that the closing/cancellation procedure + // has been completed, and finish. + segment.cleanElement(index) + completeCloseOrCancel() + return RESULT_CLOSED + } + // A waiting receiver is stored in the cell. + else -> { + assert { state is Waiter || state is WaiterEB } + // As the element will be passed directly to the waiter, + // the algorithm cleans the element slot in the cell. + segment.cleanElement(index) + // Unwrap the waiting receiver from `WaiterEB` if needed. + // As a receiver is stored in the cell, the buffer expansion + // procedure would finish, so senders simply ignore the "EB" marker. + val receiver = if (state is WaiterEB) state.waiter else state + // Try to make a rendezvous with the suspended receiver. + return if (receiver.tryResumeReceiver(element)) { + // Rendezvous! Move the cell state to `DONE_RCV` and finish. + segment.setState(index, DONE_RCV) + onReceiveDequeued() + RESULT_RENDEZVOUS + } else { + // The resumption has failed. Update the cell state correspondingly + // and clean the element field. It is also possible for a concurrent + // `expandBuffer()` or the cancellation handler to update the cell state; + // we can safely ignore these updates as senders do not help `expandBuffer()`. + if (segment.getAndSetState(index, INTERRUPTED_RCV) !== INTERRUPTED_RCV) { + segment.onCancelledRequest(index, true) + } + RESULT_FAILED + } + } + } + } + } + + /** + * Checks whether a [send] invocation is bound to suspend if it is called + * with the specified [sendersAndCloseStatus], [receivers], and [bufferEnd] + * values. When this channel is already closed, the function returns `false`. + * + * Specifically, [send] suspends if the channel is not unlimited, + * the number of receivers is greater than then index of the working cell of the + * potential [send] invocation, and the buffer does not cover this cell + * in case of buffered channel. + * When the channel is already closed, [send] does not suspend. + */ + @JsName("shouldSendSuspend0") + private fun shouldSendSuspend(curSendersAndCloseStatus: Long): Boolean { + // Does not suspend if the channel is already closed. + if (curSendersAndCloseStatus.isClosedForSend0) return false + // Does not suspend if a rendezvous may happen or the buffer is not full. + return !bufferOrRendezvousSend(curSendersAndCloseStatus.sendersCounter) + } + + /** + * Returns `true` when the specified [send] should place + * its element to the working cell without suspension. + */ + private fun bufferOrRendezvousSend(curSenders: Long): Boolean = + curSenders < bufferEndCounter || curSenders < receiversCounter + capacity + + /** + * Checks whether a [send] invocation is bound to suspend if it is called + * with the current counter and close status values. See [shouldSendSuspend] for details. + * + * Note that this implementation is _false positive_ in case of rendezvous channels, + * so it can return `false` when a [send] invocation is bound to suspend. Specifically, + * the counter of `receive()` operations may indicate that there is a waiting receiver, + * while it has already been cancelled, so the potential rendezvous is bound to fail. + */ + internal open fun shouldSendSuspend(): Boolean = shouldSendSuspend(sendersAndCloseStatus.value) + + /** + * Tries to resume this receiver with the specified [element] as a result. + * Returns `true` on success and `false` otherwise. + */ + @Suppress("UNCHECKED_CAST") + private fun Any.tryResumeReceiver(element: E): Boolean = when(this) { + is SelectInstance<*> -> { // `onReceiveXXX` select clause + trySelect(this@BufferedChannel, element) + } + is ReceiveCatching<*> -> { + this as ReceiveCatching<E> + cont.tryResume0(success(element), onUndeliveredElement?.bindCancellationFun(element, cont.context)) + } + is BufferedChannel<*>.BufferedChannelIterator -> { + this as BufferedChannel<E>.BufferedChannelIterator + tryResumeHasNext(element) + } + is CancellableContinuation<*> -> { // `receive()` + this as CancellableContinuation<E> + tryResume0(element, onUndeliveredElement?.bindCancellationFun(element, context)) + } + else -> error("Unexpected receiver type: $this") + } + + // ########################## + // # The receive operations # + // ########################## + + /** + * This function is invoked when a receiver is added as a waiter in this channel. + */ + protected open fun onReceiveEnqueued() {} + + /** + * This function is invoked when a waiting receiver is no longer stored in this channel; + * independently on whether it is caused by rendezvous, cancellation, or channel closing. + */ + protected open fun onReceiveDequeued() {} + + override suspend fun receive(): E = + receiveImpl( // <-- this is an inline function + // Do not create a continuation until it is required; + // it is created later via [onNoWaiterSuspend], if needed. + waiter = null, + // Return the received element on successful retrieval from + // the buffer or rendezvous with a suspended sender. + // Also, inform `BufferedChannel` extensions that + // synchronization of this receive operation is completed. + onElementRetrieved = { element -> + return element + }, + // As no waiter is provided, suspension is impossible. + onSuspend = { _, _, _ -> error("unexpected") }, + // Throw an exception if the channel is already closed. + onClosed = { throw recoverStackTrace(receiveException) }, + // If `receive()` decides to suspend, the corresponding + // `suspend` function that creates a continuation is called. + // The tail-call optimization is applied here. + onNoWaiterSuspend = { segm, i, r -> receiveOnNoWaiterSuspend(segm, i, r) } + ) + + private suspend fun receiveOnNoWaiterSuspend( + /* The working cell is specified by + the segment and the index in it. */ + segment: ChannelSegment<E>, + index: Int, + /* The global index of the cell. */ + r: Long + ) = suspendCancellableCoroutineReusable { cont -> + receiveImplOnNoWaiter( // <-- this is an inline function + segment = segment, index = index, r = r, + // Store the created continuation as a waiter. + waiter = cont, + // In case of successful element retrieval, resume + // the continuation with the element and inform the + // `BufferedChannel` extensions that the synchronization + // is completed. Importantly, the receiver coroutine + // may be cancelled after it is successfully resumed but + // not dispatched yet. In case `onUndeliveredElement` is + // specified, we need to invoke it in the latter case. + onElementRetrieved = { element -> + val onCancellation = onUndeliveredElement?.bindCancellationFun(element, cont.context) + cont.resume(element, onCancellation) + }, + onClosed = { onClosedReceiveOnNoWaiterSuspend(cont) }, + ) + } + + private fun Waiter.prepareReceiverForSuspension(segment: ChannelSegment<E>, index: Int) { + onReceiveEnqueued() + invokeOnCancellation(segment, index) + } + + private fun onClosedReceiveOnNoWaiterSuspend(cont: CancellableContinuation<E>) { + cont.resumeWithException(receiveException) + } + + /* + The implementation is exactly the same as of `receive()`, + with the only difference that this function returns a `ChannelResult` + instance and does not throw exception explicitly in case the channel + is already closed for receiving. Please refer the plain `receive()` + implementation for the comments. + */ + override suspend fun receiveCatching(): ChannelResult<E> = + receiveImpl( // <-- this is an inline function + waiter = null, + onElementRetrieved = { element -> + success(element) + }, + onSuspend = { _, _, _ -> error("unexpected") }, + onClosed = { closed(closeCause) }, + onNoWaiterSuspend = { segm, i, r -> receiveCatchingOnNoWaiterSuspend(segm, i, r) } + ) + + private suspend fun receiveCatchingOnNoWaiterSuspend( + segment: ChannelSegment<E>, + index: Int, + r: Long + ) = suspendCancellableCoroutineReusable { cont -> + val waiter = ReceiveCatching(cont as CancellableContinuationImpl<ChannelResult<E>>) + receiveImplOnNoWaiter( + segment, index, r, + waiter = waiter, + onElementRetrieved = { element -> + cont.resume(success(element), onUndeliveredElement?.bindCancellationFun(element, cont.context)) + }, + onClosed = { onClosedReceiveCatchingOnNoWaiterSuspend(cont) } + ) + } + + private fun onClosedReceiveCatchingOnNoWaiterSuspend(cont: CancellableContinuation<ChannelResult<E>>) { + cont.resume(closed(closeCause)) + } + + override fun tryReceive(): ChannelResult<E> { + // Read the `receivers` counter first. + val r = receivers.value + val sendersAndCloseStatusCur = sendersAndCloseStatus.value + // Is this channel closed for receive? + if (sendersAndCloseStatusCur.isClosedForReceive0) { + return closed(closeCause) + } + // Do not try to receive an element if the plain `receive()` operation would suspend. + val s = sendersAndCloseStatusCur.sendersCounter + if (r >= s) return failure() + // Let's try to retrieve an element! + // The logic is similar to the plain `receive()` operation, with + // the only difference that we store `INTERRUPTED_RCV` in case + // the operation decides to suspend. This way, we can leverage + // the unconditional `Fetch-and-Add` instruction. + // One may consider storing `INTERRUPTED_RCV` instead of an actual waiter + // on suspension (a.k.a. "no elements to retrieve") as a short-cut of + // "suspending and cancelling immediately". + return receiveImpl( // <-- this is an inline function + // Store an already interrupted receiver in case of suspension. + waiter = INTERRUPTED_RCV, + // Finish when an element is successfully retrieved. + onElementRetrieved = { element -> success(element) }, + // On suspension, the `INTERRUPTED_RCV` token has been + // installed, and this `tryReceive()` must fail. + onSuspend = { segm, _, globalIndex -> + // Emulate "cancelled" receive, thus invoking 'waitExpandBufferCompletion' manually, + // because effectively there were no cancellation + waitExpandBufferCompletion(globalIndex) + segm.onSlotCleaned() + failure() + }, + // If the channel is closed, return the corresponding result. + onClosed = { closed(closeCause) } + ) + } + + /** + * Extracts the first element from this channel until the cell with the specified + * index is moved to the logical buffer. This is a key procedure for the _conflated_ + * channel implementation, see [ConflatedBufferedChannel] with the [BufferOverflow.DROP_OLDEST] + * strategy on buffer overflowing. + */ + protected fun dropFirstElementUntilTheSpecifiedCellIsInTheBuffer(globalCellIndex: Long) { + assert { isConflatedDropOldest } + // Read the segment reference before the counter increment; + // it is crucial to be able to find the required segment later. + var segment = receiveSegment.value + while (true) { + // Read the receivers counter to check whether the specified cell is already in the buffer + // or should be moved to the buffer in a short time, due to the already started `receive()`. + val r = this.receivers.value + if (globalCellIndex < max(r + capacity, bufferEndCounter)) return + // The cell is outside the buffer. Try to extract the first element + // if the `receivers` counter has not been changed. + if (!this.receivers.compareAndSet(r, r + 1)) continue + // Count the required segment id and the cell index in it. + val id = r / SEGMENT_SIZE + val i = (r % SEGMENT_SIZE).toInt() + // Try to find the required segment if the initially obtained + // segment (in the beginning of this function) has lower id. + if (segment.id != id) { + // Find the required segment, restarting the operation if it has not been found. + segment = findSegmentReceive(id, segment) ?: + // The required segment has not been found. It is possible that the channel is already + // closed for receiving, so the linked list of segments is closed as well. + // In the latter case, the operation will finish eventually after incrementing + // the `receivers` counter sufficient times. Note that it is impossible to check + // whether this channel is closed for receiving (we do this in `receive`), + // as it may call this function when helping to complete closing the channel. + continue + } + // Update the cell according to the cell life-cycle. + val updCellResult = updateCellReceive(segment, i, r, null) + when { + updCellResult === FAILED -> { + // The cell is poisoned; restart from the beginning. + // To avoid memory leaks, we also need to reset + // the `prev` pointer of the working segment. + if (r < sendersCounter) segment.cleanPrev() + } + else -> { // element + // A buffered element was retrieved from the cell. + // Clean the reference to the previous segment. + segment.cleanPrev() + @Suppress("UNCHECKED_CAST") + onUndeliveredElement?.callUndeliveredElementCatchingException(updCellResult as E)?.let { throw it } + } + } + } + } + + /** + * Abstract receive implementation. + */ + private inline fun <R> receiveImpl( + /* The waiter to be stored in case of suspension, + or `null` if the waiter is not created yet. + In the latter case, if the algorithm decides + to suspend, [onNoWaiterSuspend] is called. */ + waiter: Any?, + /* This lambda is invoked when an element has been + successfully retrieved, either from the buffer or + by making a rendezvous with a suspended sender. */ + onElementRetrieved: (element: E) -> R, + /* This lambda is called when the operation suspends in the cell + specified by the segment and its global and in-segment indices. */ + onSuspend: (segm: ChannelSegment<E>, i: Int, r: Long) -> R, + /* This lambda is called when the channel is observed + in the closed state and no waiting sender is found, + which means that it is closed for receiving. */ + onClosed: () -> R, + /* This lambda is called when the operation decides + to suspend, but the waiter is not provided (equals `null`). + It should create a waiter and delegate to `sendImplOnNoWaiter`. */ + onNoWaiterSuspend: ( + segm: ChannelSegment<E>, + i: Int, + r: Long + ) -> R = { _, _, _ -> error("unexpected") } + ): R { + // Read the segment reference before the counter increment; + // it is crucial to be able to find the required segment later. + var segment = receiveSegment.value + while (true) { + // Similar to the `send(e)` operation, `receive()` first checks + // whether the channel is already closed for receiving. + if (isClosedForReceive) return onClosed() + // Atomically increments the `receivers` counter + // and obtain the value right before the increment. + val r = this.receivers.getAndIncrement() + // Count the required segment id and the cell index in it. + val id = r / SEGMENT_SIZE + val i = (r % SEGMENT_SIZE).toInt() + // Try to find the required segment if the initially obtained + // segment (in the beginning of this function) has lower id. + if (segment.id != id) { + // Find the required segment, restarting the operation if it has not been found. + segment = findSegmentReceive(id, segment) ?: + // The required segment is not found. It is possible that the channel is already + // closed for receiving, so the linked list of segments is closed as well. + // In the latter case, the operation fails with the corresponding check at the beginning. + continue + } + // Update the cell according to the cell life-cycle. + val updCellResult = updateCellReceive(segment, i, r, waiter) + return when { + updCellResult === SUSPEND -> { + // The operation has decided to suspend and + // stored the specified waiter in the cell. + (waiter as? Waiter)?.prepareReceiverForSuspension(segment, i) + onSuspend(segment, i, r) + } + updCellResult === FAILED -> { + // The operation has tried to make a rendezvous + // but failed: either the opposite request has + // already been cancelled or the cell is poisoned. + // Restart from the beginning in this case. + // To avoid memory leaks, we also need to reset + // the `prev` pointer of the working segment. + if (r < sendersCounter) segment.cleanPrev() + continue + } + updCellResult === SUSPEND_NO_WAITER -> { + // The operation has decided to suspend, + // but no waiter has been provided. + onNoWaiterSuspend(segment, i, r) + } + else -> { // element + // Either a buffered element was retrieved from the cell + // or a rendezvous with a waiting sender has happened. + // Clean the reference to the previous segment before finishing. + segment.cleanPrev() + @Suppress("UNCHECKED_CAST") + onElementRetrieved(updCellResult as E) + } + } + } + } + + private inline fun receiveImplOnNoWaiter( + /* The working cell is specified by + the segment and the index in it. */ + segment: ChannelSegment<E>, + index: Int, + /* The global index of the cell. */ + r: Long, + /* The waiter to be stored in case of suspension. */ + waiter: Waiter, + /* This lambda is invoked when an element has been + successfully retrieved, either from the buffer or + by making a rendezvous with a suspended sender. */ + onElementRetrieved: (element: E) -> Unit, + /* This lambda is called when the channel is observed + in the closed state and no waiting senders is found, + which means that it is closed for receiving. */ + onClosed: () -> Unit + ) { + // Update the cell with the non-null waiter, + // restarting from the beginning on failure. + // Check the `receiveImpl(..)` function for the comments. + val updCellResult = updateCellReceive(segment, index, r, waiter) + when { + updCellResult === SUSPEND -> { + waiter.prepareReceiverForSuspension(segment, index) + } + updCellResult === FAILED -> { + if (r < sendersCounter) segment.cleanPrev() + receiveImpl( + waiter = waiter, + onElementRetrieved = onElementRetrieved, + onSuspend = { _, _, _ -> }, + onClosed = onClosed + ) + } + else -> { + segment.cleanPrev() + @Suppress("UNCHECKED_CAST") + onElementRetrieved(updCellResult as E) + } + } + } + + private fun updateCellReceive( + /* The working cell is specified by + the segment and the index in it. */ + segment: ChannelSegment<E>, + index: Int, + /* The global index of the cell. */ + r: Long, + /* The waiter to be stored in case of suspension. */ + waiter: Any?, + ): Any? { + // This is a fast-path of `updateCellReceiveSlow(..)`. + // + // Read the current cell state. + val state = segment.getState(index) + when { + // The cell is empty. + state === null -> { + // If a rendezvous must happen, the operation does not wait + // until the cell stores a buffered element or a suspended + // sender, poisoning the cell and restarting instead. + // Otherwise, try to store the specified waiter in the cell. + val senders = sendersAndCloseStatus.value.sendersCounter + if (r >= senders) { + // This `receive()` operation should suspend. + if (waiter === null) { + // The waiter is not specified; + // return the corresponding result. + return SUSPEND_NO_WAITER + } + // Try to install the waiter. + if (segment.casState(index, state, waiter)) { + // The waiter has been successfully installed. + // Invoke the `expandBuffer()` procedure and finish. + expandBuffer() + return SUSPEND + } + } + } + // The cell stores a buffered element. + state === BUFFERED -> if (segment.casState(index, state, DONE_RCV)) { + // Retrieve the element and expand the buffer. + expandBuffer() + return segment.retrieveElement(index) + } + } + return updateCellReceiveSlow(segment, index, r, waiter) + } + + private fun updateCellReceiveSlow( + /* The working cell is specified by + the segment and the index in it. */ + segment: ChannelSegment<E>, + index: Int, + /* The global index of the cell. */ + r: Long, + /* The waiter to be stored in case of suspension. */ + waiter: Any?, + ): Any? { + // The cell state should be updated according to its state machine; + // see the paper mentioned in the very beginning for the algorithm details. + while (true) { + // Read the current cell state. + val state = segment.getState(index) + when { + // The cell is empty. + state === null || state === IN_BUFFER -> { + // If a rendezvous must happen, the operation does not wait + // until the cell stores a buffered element or a suspended + // sender, poisoning the cell and restarting instead. + // Otherwise, try to store the specified waiter in the cell. + val senders = sendersAndCloseStatus.value.sendersCounter + if (r < senders) { + // The cell is already covered by sender, + // so a rendezvous must happen. Unfortunately, + // the cell is empty, so the operation poisons it. + if (segment.casState(index, state, POISONED)) { + // When the cell becomes poisoned, it is essentially + // the same as storing an already cancelled receiver. + // Thus, the `expandBuffer()` procedure should be invoked. + expandBuffer() + return FAILED + } + } else { + // This `receive()` operation should suspend. + if (waiter === null) { + // The waiter is not specified; + // return the corresponding result. + return SUSPEND_NO_WAITER + } + // Try to install the waiter. + if (segment.casState(index, state, waiter)) { + // The waiter has been successfully installed. + // Invoke the `expandBuffer()` procedure and finish. + expandBuffer() + return SUSPEND + } + } + } + // The cell stores a buffered element. + state === BUFFERED -> if (segment.casState(index, state, DONE_RCV)) { + // Retrieve the element and expand the buffer. + expandBuffer() + return segment.retrieveElement(index) + } + // The cell stores an interrupted sender. + state === INTERRUPTED_SEND -> return FAILED + // The cell is already poisoned by a concurrent + // `hasElements` call. Restart in this case. + state === POISONED -> return FAILED + // This channel is already closed. + state === CHANNEL_CLOSED -> { + // Although the channel is closed, it is still required + // to call the `expandBuffer()` procedure to keep + // `waitForExpandBufferCompletion()` correct. + expandBuffer() + return FAILED + } + // A concurrent `expandBuffer()` is resuming a + // suspended sender. Wait in a spin-loop until + // the resumption attempt completes: the cell + // state must change to either `BUFFERED` or + // `INTERRUPTED_SEND`. + state === RESUMING_BY_EB -> continue + // The cell stores a suspended sender; try to resume it. + else -> { + // To synchronize with expandBuffer(), the algorithm + // first moves the cell to an intermediate `S_RESUMING_BY_RCV` + // state, updating it to either `BUFFERED` (on success) or + // `INTERRUPTED_SEND` (on failure). + if (segment.casState(index, state, RESUMING_BY_RCV)) { + // Has a concurrent `expandBuffer()` delegated its completion? + val helpExpandBuffer = state is WaiterEB + // Extract the sender if needed and try to resume it. + val sender = if (state is WaiterEB) state.waiter else state + return if (sender.tryResumeSender(segment, index)) { + // The sender has been resumed successfully! + // Update the cell state correspondingly, + // expand the buffer, and return the element + // stored in the cell. + // In case a concurrent `expandBuffer()` has delegated + // its completion, the procedure should finish, as the + // sender is resumed. Thus, no further action is required. + segment.setState(index, DONE_RCV) + expandBuffer() + segment.retrieveElement(index) + } else { + // The resumption has failed. Update the cell correspondingly. + // In case a concurrent `expandBuffer()` has delegated + // its completion, the procedure should skip this cell, so + // `expandBuffer()` should be called once again. + segment.setState(index, INTERRUPTED_SEND) + segment.onCancelledRequest(index, false) + if (helpExpandBuffer) expandBuffer() + FAILED + } + } + } + } + } + } + + private fun Any.tryResumeSender(segment: ChannelSegment<E>, index: Int): Boolean = when (this) { + is CancellableContinuation<*> -> { // suspended `send(e)` operation + @Suppress("UNCHECKED_CAST") + this as CancellableContinuation<Unit> + tryResume0(Unit) + } + is SelectInstance<*> -> { + this as SelectImplementation<*> + val trySelectResult = trySelectDetailed(clauseObject = this@BufferedChannel, result = Unit) + // Clean the element slot to avoid memory leaks + // if this `select` clause should be re-registered. + if (trySelectResult === REREGISTER) segment.cleanElement(index) + // Was the resumption successful? + trySelectResult === SUCCESSFUL + } + is SendBroadcast -> cont.tryResume0(true) // // suspended `sendBroadcast(e)` operation + else -> error("Unexpected waiter: $this") + } + + // ################################ + // # The expandBuffer() procedure # + // ################################ + + private fun expandBuffer() { + // Do not need to take any action if + // this channel is rendezvous or unlimited. + if (isRendezvousOrUnlimited) return + // Read the current segment of + // the `expandBuffer()` procedure. + var segment = bufferEndSegment.value + // Try to expand the buffer until succeed. + try_again@ while (true) { + // Increment the logical end of the buffer. + // The `b`-th cell is going to be added to the buffer. + val b = bufferEnd.getAndIncrement() + val id = b / SEGMENT_SIZE + // After that, read the current `senders` counter. + // In case its value is lower than `b`, the `send(e)` + // invocation that will work with this `b`-th cell + // will detect that the cell is already a part of the + // buffer when comparing with the `bufferEnd` counter. + // However, `bufferEndSegment` may reference an outdated + // segment, which should be updated to avoid memory leaks. + val s = sendersCounter + if (s <= b) { + // Should `bufferEndSegment` be moved forward to avoid memory leaks? + if (segment.id < id && segment.next != null) + moveSegmentBufferEndToSpecifiedOrLast(id, segment) + // Increment the number of completed `expandBuffer()`-s and finish. + incCompletedExpandBufferAttempts() + return + } + // Is `bufferEndSegment` outdated or is the segment with the required id already removed? + // Find the required segment, creating new ones if needed. + if (segment.id != id) { + segment = findSegmentBufferEnd(id, segment, b) + // Restart if the required segment is removed, or + // the linked list of segments is already closed, + // and the required one will never be created. + // Please note that `findSegmentBuffer(..)` updates + // the number of completed `expandBuffer()` attempt + // in this case. + ?: continue@try_again + } + // Try to add the cell to the logical buffer, + // updating the cell state according to the state-machine. + val i = (b % SEGMENT_SIZE).toInt() + if (updateCellExpandBuffer(segment, i, b)) { + // The cell has been added to the logical buffer! + // Increment the number of completed `expandBuffer()`-s and finish. + // + // Note that it is possible to increment the number of + // completed `expandBuffer()` attempts earlier, right + // after the segment is obtained. We find this change + // counter-intuitive and prefer to avoid it. + incCompletedExpandBufferAttempts() + return + } else { + // The cell has not been added to the buffer. + // Increment the number of completed `expandBuffer()` + // attempts and restart. + incCompletedExpandBufferAttempts() + continue@try_again + } + } + } + + private fun updateCellExpandBuffer( + /* The working cell is specified by + the segment and the index in it. */ + segment: ChannelSegment<E>, + index: Int, + /* The global index of the cell. */ + b: Long + ): Boolean { + // This is a fast-path of `updateCellExpandBufferSlow(..)`. + // + // Read the current cell state. + val state = segment.getState(index) + if (state is Waiter) { + // Usually, a sender is stored in the cell. + // However, it is possible for a concurrent + // receiver to be already suspended there. + // Try to distinguish whether the waiter is a + // sender by comparing the global cell index with + // the `receivers` counter. In case the cell is not + // covered by a receiver, a sender is stored in the cell. + if (b >= receivers.value) { + // The cell stores a suspended sender. Try to resume it. + // To synchronize with a concurrent `receive()`, the algorithm + // first moves the cell state to an intermediate `RESUMING_BY_EB` + // state, updating it to either `BUFFERED` (on successful resumption) + // or `INTERRUPTED_SEND` (on failure). + if (segment.casState(index, state, RESUMING_BY_EB)) { + return if (state.tryResumeSender(segment, index)) { + // The sender has been resumed successfully! + // Move the cell to the logical buffer and finish. + segment.setState(index, BUFFERED) + true + } else { + // The resumption has failed. + segment.setState(index, INTERRUPTED_SEND) + segment.onCancelledRequest(index, false) + false + } + } + } + } + return updateCellExpandBufferSlow(segment, index, b) + } + + private fun updateCellExpandBufferSlow( + /* The working cell is specified by + the segment and the index in it. */ + segment: ChannelSegment<E>, + index: Int, + /* The global index of the cell. */ + b: Long + ): Boolean { + // Update the cell state according to its state machine. + // See the paper mentioned in the very beginning for + // the cell life-cycle and the algorithm details. + while (true) { + // Read the current cell state. + val state = segment.getState(index) + when { + // A suspended waiter, sender or receiver. + state is Waiter -> { + // Usually, a sender is stored in the cell. + // However, it is possible for a concurrent + // receiver to be already suspended there. + // Try to distinguish whether the waiter is a + // sender by comparing the global cell index with + // the `receivers` counter. In case the cell is not + // covered by a receiver, a sender is stored in the cell. + if (b < receivers.value) { + // The algorithm cannot distinguish whether the + // suspended in the cell operation is sender or receiver. + // To make progress, `expandBuffer()` delegates its completion + // to an upcoming pairwise request, atomically wrapping + // the waiter in `WaiterEB`. In case a sender is stored + // in the cell, the upcoming receiver will call `expandBuffer()` + // if the sender resumption fails; thus, effectively, skipping + // this cell. Otherwise, if a receiver is stored in the cell, + // this `expandBuffer()` procedure must finish; therefore, + // sender ignore the `WaiterEB` wrapper. + if (segment.casState(index, state, WaiterEB(waiter = state))) + return true + } else { + // The cell stores a suspended sender. Try to resume it. + // To synchronize with a concurrent `receive()`, the algorithm + // first moves the cell state to an intermediate `RESUMING_BY_EB` + // state, updating it to either `BUFFERED` (on successful resumption) + // or `INTERRUPTED_SEND` (on failure). + if (segment.casState(index, state, RESUMING_BY_EB)) { + return if (state.tryResumeSender(segment, index)) { + // The sender has been resumed successfully! + // Move the cell to the logical buffer and finish. + segment.setState(index, BUFFERED) + true + } else { + // The resumption has failed. + segment.setState(index, INTERRUPTED_SEND) + segment.onCancelledRequest(index, false) + false + } + } + } + } + // The cell stores an interrupted sender, skip it. + state === INTERRUPTED_SEND -> return false + // The cell is empty, a concurrent sender is coming. + state === null -> { + // To inform a concurrent sender that this cell is + // already a part of the buffer, the algorithm moves + // it to a special `IN_BUFFER` state. + if (segment.casState(index, state, IN_BUFFER)) return true + } + // The cell is already a part of the buffer, finish. + state === BUFFERED -> return true + // The cell is already processed by a receiver, no further action is required. + state === POISONED || state === DONE_RCV || state === INTERRUPTED_RCV -> return true + // The channel is closed, all the following + // cells are already in the same state, finish. + state === CHANNEL_CLOSED -> return true + // A concurrent receiver is resuming the suspended sender. + // Wait in a spin-loop until it changes the cell state + // to either `DONE_RCV` or `INTERRUPTED_SEND`. + state === RESUMING_BY_RCV -> continue // spin wait + else -> error("Unexpected cell state: $state") + } + } + } + + /** + * Increments the counter of completed [expandBuffer] invocations. + * To guarantee starvation-freedom for [waitExpandBufferCompletion], + * which waits until the counters of started and completed [expandBuffer] calls + * coincide and become greater or equal to the specified value, + * [waitExpandBufferCompletion] may set a flag that pauses further progress. + */ + private fun incCompletedExpandBufferAttempts(nAttempts: Long = 1) { + // Increment the number of completed `expandBuffer()` calls. + completedExpandBuffersAndPauseFlag.addAndGet(nAttempts).also { + // Should further `expandBuffer()`-s be paused? + // If so, this thread should wait in a spin-loop + // until the flag is unset. + if (it.ebPauseExpandBuffers) { + @Suppress("ControlFlowWithEmptyBody") + while (completedExpandBuffersAndPauseFlag.value.ebPauseExpandBuffers) {} + } + } + } + + /** + * Waits in a spin-loop until the [expandBuffer] call that + * should process the [globalIndex]-th cell is completed. + * Essentially, it waits until the numbers of started ([bufferEnd]) + * and completed ([completedExpandBuffersAndPauseFlag]) [expandBuffer] + * attempts coincide and become equal or greater than [globalIndex]. + * To avoid starvation, this function may set a flag + * that pauses further progress. + */ + internal fun waitExpandBufferCompletion(globalIndex: Long) { + // Do nothing if this channel is rendezvous or unlimited; + // `expandBuffer()` is not used in these cases. + if (isRendezvousOrUnlimited) return + // Wait in an infinite loop until the number of started + // buffer expansion calls become not lower than the cell index. + @Suppress("ControlFlowWithEmptyBody") + while (bufferEndCounter <= globalIndex) {} + // Now it is guaranteed that the `expandBuffer()` call that + // should process the required cell has been started. + // Wait in a fixed-size spin-loop until the numbers of + // started and completed buffer expansion calls coincide. + repeat(EXPAND_BUFFER_COMPLETION_WAIT_ITERATIONS) { + // Read the number of started buffer expansion calls. + val b = bufferEndCounter + // Read the number of completed buffer expansion calls. + val ebCompleted = completedExpandBuffersAndPauseFlag.value.ebCompletedCounter + // Do the numbers of started and completed calls coincide? + // Note that we need to re-read the number of started `expandBuffer()` + // calls to obtain a correct snapshot. + // Here we wait to a precise match in order to ensure that **our matching expandBuffer()** + // completed. The only way to ensure that is to check that number of started expands == number of finished expands + if (b == ebCompleted && b == bufferEndCounter) return + } + // To avoid starvation, pause further `expandBuffer()` calls. + completedExpandBuffersAndPauseFlag.update { + constructEBCompletedAndPauseFlag(it.ebCompletedCounter, true) + } + // Now wait in an infinite spin-loop until the counters coincide. + while (true) { + // Read the number of started buffer expansion calls. + val b = bufferEndCounter + // Read the number of completed buffer expansion calls + // along with the flag that pauses further progress. + val ebCompletedAndBit = completedExpandBuffersAndPauseFlag.value + val ebCompleted = ebCompletedAndBit.ebCompletedCounter + val pauseExpandBuffers = ebCompletedAndBit.ebPauseExpandBuffers + // Do the numbers of started and completed calls coincide? + // Note that we need to re-read the number of started `expandBuffer()` + // calls to obtain a correct snapshot. + if (b == ebCompleted && b == bufferEndCounter) { + // Unset the flag, which pauses progress, and finish. + completedExpandBuffersAndPauseFlag.update { + constructEBCompletedAndPauseFlag(it.ebCompletedCounter, false) + } + return + } + // It is possible that a concurrent caller of this function + // has unset the flag, which pauses further progress to avoid + // starvation. In this case, set the flag back. + if (!pauseExpandBuffers) { + completedExpandBuffersAndPauseFlag.compareAndSet( + ebCompletedAndBit, + constructEBCompletedAndPauseFlag(ebCompleted, true) + ) + } + } + } + + + // ####################### + // ## Select Expression ## + // ####################### + + @Suppress("UNCHECKED_CAST") + override val onSend: SelectClause2<E, BufferedChannel<E>> + get() = SelectClause2Impl( + clauseObject = this@BufferedChannel, + regFunc = BufferedChannel<*>::registerSelectForSend as RegistrationFunction, + processResFunc = BufferedChannel<*>::processResultSelectSend as ProcessResultFunction + ) + + @Suppress("UNCHECKED_CAST") + protected open fun registerSelectForSend(select: SelectInstance<*>, element: Any?) = + sendImpl( // <-- this is an inline function + element = element as E, + waiter = select, + onRendezvousOrBuffered = { select.selectInRegistrationPhase(Unit) }, + onSuspend = { _, _ -> }, + onClosed = { onClosedSelectOnSend(element, select) } + ) + + + private fun onClosedSelectOnSend(element: E, select: SelectInstance<*>) { + onUndeliveredElement?.callUndeliveredElement(element, select.context) + select.selectInRegistrationPhase(CHANNEL_CLOSED) + } + + @Suppress("UNUSED_PARAMETER", "RedundantNullableReturnType") + private fun processResultSelectSend(ignoredParam: Any?, selectResult: Any?): Any? = + if (selectResult === CHANNEL_CLOSED) throw sendException + else this + + @Suppress("UNCHECKED_CAST") + override val onReceive: SelectClause1<E> + get() = SelectClause1Impl( + clauseObject = this@BufferedChannel, + regFunc = BufferedChannel<*>::registerSelectForReceive as RegistrationFunction, + processResFunc = BufferedChannel<*>::processResultSelectReceive as ProcessResultFunction, + onCancellationConstructor = onUndeliveredElementReceiveCancellationConstructor + ) + + @Suppress("UNCHECKED_CAST") + override val onReceiveCatching: SelectClause1<ChannelResult<E>> + get() = SelectClause1Impl( + clauseObject = this@BufferedChannel, + regFunc = BufferedChannel<*>::registerSelectForReceive as RegistrationFunction, + processResFunc = BufferedChannel<*>::processResultSelectReceiveCatching as ProcessResultFunction, + onCancellationConstructor = onUndeliveredElementReceiveCancellationConstructor + ) + + @Suppress("OVERRIDE_DEPRECATION", "UNCHECKED_CAST") + override val onReceiveOrNull: SelectClause1<E?> + get() = SelectClause1Impl( + clauseObject = this@BufferedChannel, + regFunc = BufferedChannel<*>::registerSelectForReceive as RegistrationFunction, + processResFunc = BufferedChannel<*>::processResultSelectReceiveOrNull as ProcessResultFunction, + onCancellationConstructor = onUndeliveredElementReceiveCancellationConstructor + ) + + @Suppress("UNUSED_PARAMETER") + private fun registerSelectForReceive(select: SelectInstance<*>, ignoredParam: Any?) = + receiveImpl( // <-- this is an inline function + waiter = select, + onElementRetrieved = { elem -> select.selectInRegistrationPhase(elem) }, + onSuspend = { _, _, _ -> }, + onClosed = { onClosedSelectOnReceive(select) } + ) + + private fun onClosedSelectOnReceive(select: SelectInstance<*>) { + select.selectInRegistrationPhase(CHANNEL_CLOSED) + } + + @Suppress("UNUSED_PARAMETER") + private fun processResultSelectReceive(ignoredParam: Any?, selectResult: Any?): Any? = + if (selectResult === CHANNEL_CLOSED) throw receiveException + else selectResult + + @Suppress("UNUSED_PARAMETER") + private fun processResultSelectReceiveOrNull(ignoredParam: Any?, selectResult: Any?): Any? = + if (selectResult === CHANNEL_CLOSED) { + if (closeCause == null) null + else throw receiveException + } else selectResult + + @Suppress("UNCHECKED_CAST", "UNUSED_PARAMETER", "RedundantNullableReturnType") + private fun processResultSelectReceiveCatching(ignoredParam: Any?, selectResult: Any?): Any? = + if (selectResult === CHANNEL_CLOSED) closed(closeCause) + else success(selectResult as E) + + @Suppress("UNCHECKED_CAST") + private val onUndeliveredElementReceiveCancellationConstructor: OnCancellationConstructor? = onUndeliveredElement?.let { + { select: SelectInstance<*>, _: Any?, element: Any? -> + { if (element !== CHANNEL_CLOSED) onUndeliveredElement.callUndeliveredElement(element as E, select.context) } + } + } + + // ###################### + // ## Iterator Support ## + // ###################### + + override fun iterator(): ChannelIterator<E> = BufferedChannelIterator() + + /** + * The key idea is that an iterator is a special receiver type, + * which should be resumed differently to [receive] and [onReceive] + * operations, but can be served as a waiter in a way similar to + * [CancellableContinuation] and [SelectInstance]. + * + * Roughly, [hasNext] is a [receive] sibling, while [next] simply + * returns the already retrieved element. From the implementation + * side, [receiveResult] stores the element retrieved by [hasNext] + * (or a special [CHANNEL_CLOSED] token if the channel is closed). + * + * The [invoke] function is a [CancelHandler] implementation, + * which requires knowing the [segment] and the [index] in it + * that specify the location of the stored iterator. + * + * To resume the suspended [hasNext] call, a special [tryResumeHasNext] + * function should be used in a way similar to [CancellableContinuation.tryResume] + * and [SelectInstance.trySelect]. When the channel becomes closed, + * [tryResumeHasNextOnClosedChannel] should be used instead. + */ + private inner class BufferedChannelIterator : ChannelIterator<E>, Waiter { + /** + * Stores the element retrieved by [hasNext] or + * a special [CHANNEL_CLOSED] token if this channel is closed. + * If [hasNext] has not been invoked yet, [NO_RECEIVE_RESULT] is stored. + */ + private var receiveResult: Any? = NO_RECEIVE_RESULT + + /** + * When [hasNext] suspends, this field stores the corresponding + * continuation. The [tryResumeHasNext] and [tryResumeHasNextOnClosedChannel] + * function resume this continuation when the [hasNext] invocation should complete. + */ + private var continuation: CancellableContinuationImpl<Boolean>? = null + + // `hasNext()` is just a special receive operation. + override suspend fun hasNext(): Boolean = + receiveImpl( // <-- this is an inline function + // Do not create a continuation until it is required; + // it is created later via [onNoWaiterSuspend], if needed. + waiter = null, + // Store the received element in `receiveResult` on successful + // retrieval from the buffer or rendezvous with a suspended sender. + // Also, inform the `BufferedChannel` extensions that + // the synchronization of this receive operation is completed. + onElementRetrieved = { element -> + this.receiveResult = element + true + }, + // As no waiter is provided, suspension is impossible. + onSuspend = { _, _, _ -> error("unreachable") }, + // Return `false` or throw an exception if the channel is already closed. + onClosed = { onClosedHasNext() }, + // If `hasNext()` decides to suspend, the corresponding + // `suspend` function that creates a continuation is called. + // The tail-call optimization is applied here. + onNoWaiterSuspend = { segm, i, r -> return hasNextOnNoWaiterSuspend(segm, i, r) } + ) + + private fun onClosedHasNext(): Boolean { + this.receiveResult = CHANNEL_CLOSED + val cause = closeCause ?: return false + throw recoverStackTrace(cause) + } + + private suspend fun hasNextOnNoWaiterSuspend( + /* The working cell is specified by + the segment and the index in it. */ + segment: ChannelSegment<E>, + index: Int, + /* The global index of the cell. */ + r: Long + ): Boolean = suspendCancellableCoroutineReusable { cont -> + this.continuation = cont + receiveImplOnNoWaiter( // <-- this is an inline function + segment = segment, index = index, r = r, + waiter = this, // store this iterator as a waiter + // In case of successful element retrieval, store + // it in `receiveResult` and resume the continuation. + // Importantly, the receiver coroutine may be cancelled + // after it is successfully resumed but not dispatched yet. + // In case `onUndeliveredElement` is present, we must + // invoke it in the latter case. + onElementRetrieved = { element -> + this.receiveResult = element + this.continuation = null + cont.resume(true, onUndeliveredElement?.bindCancellationFun(element, cont.context)) + }, + onClosed = { onClosedHasNextNoWaiterSuspend() } + ) + } + + override fun invokeOnCancellation(segment: Segment<*>, index: Int) { + this.continuation?.invokeOnCancellation(segment, index) + } + + private fun onClosedHasNextNoWaiterSuspend() { + // Read the current continuation and clean + // the corresponding field to avoid memory leaks. + val cont = this.continuation!! + this.continuation = null + // Update the `hasNext()` internal result. + this.receiveResult = CHANNEL_CLOSED + // If this channel was closed without exception, + // `hasNext()` should return `false`; otherwise, + // it throws the closing exception. + val cause = closeCause + if (cause == null) { + cont.resume(false) + } else { + cont.resumeWithException(recoverStackTrace(cause, cont)) + } + } + + @Suppress("UNCHECKED_CAST") + override fun next(): E { + // Read the already received result, or [NO_RECEIVE_RESULT] if [hasNext] has not been invoked yet. + val result = receiveResult + check(result !== NO_RECEIVE_RESULT) { "`hasNext()` has not been invoked" } + receiveResult = NO_RECEIVE_RESULT + // Is this channel closed? + if (result === CHANNEL_CLOSED) throw recoverStackTrace(receiveException) + // Return the element. + return result as E + } + + fun tryResumeHasNext(element: E): Boolean { + // Read the current continuation and clean + // the corresponding field to avoid memory leaks. + val cont = this.continuation!! + this.continuation = null + // Store the retrieved element in `receiveResult`. + this.receiveResult = element + // Try to resume this `hasNext()`. Importantly, the receiver coroutine + // may be cancelled after it is successfully resumed but not dispatched yet. + // In case `onUndeliveredElement` is specified, we need to invoke it in the latter case. + return cont.tryResume0(true, onUndeliveredElement?.bindCancellationFun(element, cont.context)) + } + + fun tryResumeHasNextOnClosedChannel() { + // Read the current continuation and clean + // the corresponding field to avoid memory leaks. + val cont = this.continuation!! + this.continuation = null + // Update the `hasNext()` internal result and inform + // `BufferedChannel` extensions that synchronization + // of this receive operation is completed. + this.receiveResult = CHANNEL_CLOSED + // If this channel was closed without exception, + // `hasNext()` should return `false`; otherwise, + // it throws the closing exception. + val cause = closeCause + if (cause == null) { + cont.resume(false) + } else { + cont.resumeWithException(recoverStackTrace(cause, cont)) + } + } + } + + // ############################## + // ## Closing and Cancellation ## + // ############################## + + /** + * Store the cause of closing this channel, either via [close] or [cancel] call. + * The closing cause can be set only once. + */ + private val _closeCause = atomic<Any?>(NO_CLOSE_CAUSE) + // Should be called only if this channel is closed or cancelled. + protected val closeCause get() = _closeCause.value as Throwable? + + /** Returns the closing cause if it is non-null, or [ClosedSendChannelException] otherwise. */ + protected val sendException get() = closeCause ?: ClosedSendChannelException(DEFAULT_CLOSE_MESSAGE) + + /** Returns the closing cause if it is non-null, or [ClosedReceiveChannelException] otherwise. */ + private val receiveException get() = closeCause ?: ClosedReceiveChannelException(DEFAULT_CLOSE_MESSAGE) + + /** + Stores the closed handler installed by [invokeOnClose]. + To synchronize [invokeOnClose] and [close], two additional + marker states, [CLOSE_HANDLER_INVOKED] and [CLOSE_HANDLER_CLOSED] + are used. The resulting state diagram is presented below. + + +------+ install handler +---------+ close(..) +---------+ + | null |------------------>| handler |------------>| INVOKED | + +------+ +---------+ +---------+ + | + | close(..) +--------+ + +----------->| CLOSED | + +--------+ + */ + private val closeHandler = atomic<Any?>(null) + + /** + * Invoked when channel is closed as the last action of [close] invocation. + * This method should be idempotent and can be called multiple times. + */ + protected open fun onClosedIdempotent() {} + + override fun close(cause: Throwable?): Boolean = + closeOrCancelImpl(cause, cancel = false) + + @Suppress("OVERRIDE_DEPRECATION") + final override fun cancel(cause: Throwable?): Boolean = cancelImpl(cause) + + @Suppress("OVERRIDE_DEPRECATION") + final override fun cancel() { cancelImpl(null) } + + final override fun cancel(cause: CancellationException?) { cancelImpl(cause) } + + internal open fun cancelImpl(cause: Throwable?): Boolean = + closeOrCancelImpl(cause ?: CancellationException("Channel was cancelled"), cancel = true) + + /** + * This is a common implementation for [close] and [cancel]. It first tries + * to install the specified cause; the invocation that successfully installs + * the cause returns `true` as a results of this function, while all further + * [close] and [cancel] calls return `false`. + * + * After the closing/cancellation cause is installed, the channel should be marked + * as closed or cancelled, which bounds further `send(e)`-s to fails. + * + * Then, [completeCloseOrCancel] is called, which cancels waiting `receive()` + * requests ([cancelSuspendedReceiveRequests]) and removes unprocessed elements + * ([removeUnprocessedElements]) in case this channel is cancelled. + * + * Finally, if this [closeOrCancelImpl] has installed the cause, therefore, + * has closed the channel, [closeHandler] and [onClosedIdempotent] should be invoked. + */ + protected open fun closeOrCancelImpl(cause: Throwable?, cancel: Boolean): Boolean { + // If this is a `cancel(..)` invocation, set a bit that the cancellation + // has been started. This is crucial for ensuring linearizability, + // when concurrent `close(..)` and `isClosedFor[Send,Receive]` operations + // help this `cancel(..)`. + if (cancel) markCancellationStarted() + // Try to install the specified cause. On success, this invocation will + // return `true` as a result; otherwise, it will complete with `false`. + val closedByThisOperation = _closeCause.compareAndSet(NO_CLOSE_CAUSE, cause) + // Mark this channel as closed or cancelled, depending on this operation type. + if (cancel) markCancelled() else markClosed() + // Complete the closing or cancellation procedure. + completeCloseOrCancel() + // Finally, if this operation has installed the cause, + // it should invoke the close handlers. + return closedByThisOperation.also { + onClosedIdempotent() + if (it) invokeCloseHandler() + } + } + + /** + * Invokes the installed close handler, + * updating the [closeHandler] state correspondingly. + */ + private fun invokeCloseHandler() { + val closeHandler = closeHandler.getAndUpdate { + if (it === null) { + // Inform concurrent `invokeOnClose` + // that this channel is already closed. + CLOSE_HANDLER_CLOSED + } else { + // Replace the handler with a special + // `INVOKED` marker to avoid memory leaks. + CLOSE_HANDLER_INVOKED + } + } ?: return // no handler was installed, finish. + // Invoke the handler. + @Suppress("UNCHECKED_CAST") + closeHandler as (cause: Throwable?) -> Unit + closeHandler(closeCause) + } + + override fun invokeOnClose(handler: (cause: Throwable?) -> Unit) { + // Try to install the handler, finishing on success. + if (closeHandler.compareAndSet(null, handler)) { + // Handler has been successfully set, finish the operation. + return + } + // Either another handler is already set, or this channel is closed. + // In the latter case, the current handler should be invoked. + // However, the implementation must ensure that at most one + // handler is called, throwing an `IllegalStateException` + // if another close handler has been invoked. + closeHandler.loop { cur -> + when { + cur === CLOSE_HANDLER_CLOSED -> { + // Try to update the state from `CLOSED` to `INVOKED`. + // This is crucial to guarantee that at most one handler can be called. + // On success, invoke the handler and finish. + if (closeHandler.compareAndSet(CLOSE_HANDLER_CLOSED, CLOSE_HANDLER_INVOKED)) { + handler(closeCause) + return + } + } + cur === CLOSE_HANDLER_INVOKED -> error("Another handler was already registered and successfully invoked") + else -> error("Another handler is already registered: $cur") + } + } + } + + /** + * Marks this channel as closed. + * In case [cancelImpl] has already been invoked, + * and this channel is marked with [CLOSE_STATUS_CANCELLATION_STARTED], + * this function marks the channel as cancelled. + * + * All operation that notice this channel in the closed state, + * must help to complete the closing via [completeCloseOrCancel]. + */ + private fun markClosed(): Unit = + sendersAndCloseStatus.update { cur -> + when (cur.sendersCloseStatus) { + CLOSE_STATUS_ACTIVE -> // the channel is neither closed nor cancelled + constructSendersAndCloseStatus(cur.sendersCounter, CLOSE_STATUS_CLOSED) + CLOSE_STATUS_CANCELLATION_STARTED -> // the channel is going to be cancelled + constructSendersAndCloseStatus(cur.sendersCounter, CLOSE_STATUS_CANCELLED) + else -> return // the channel is already marked as closed or cancelled. + } + } + + /** + * Marks this channel as cancelled. + * + * All operation that notice this channel in the cancelled state, + * must help to complete the cancellation via [completeCloseOrCancel]. + */ + private fun markCancelled(): Unit = + sendersAndCloseStatus.update { cur -> + constructSendersAndCloseStatus(cur.sendersCounter, CLOSE_STATUS_CANCELLED) + } + + /** + * When the cancellation procedure starts, it is critical + * to mark the closing status correspondingly. Thus, other + * operations, which may help to complete the cancellation, + * always correctly update the status to `CANCELLED`. + */ + private fun markCancellationStarted(): Unit = + sendersAndCloseStatus.update { cur -> + if (cur.sendersCloseStatus == CLOSE_STATUS_ACTIVE) + constructSendersAndCloseStatus(cur.sendersCounter, CLOSE_STATUS_CANCELLATION_STARTED) + else return // this channel is already closed or cancelled + } + + /** + * Completes the started [close] or [cancel] procedure. + */ + private fun completeCloseOrCancel() { + isClosedForSend // must finish the started close/cancel if one is detected. + } + + protected open val isConflatedDropOldest get() = false + + /** + * Completes the channel closing procedure. + */ + private fun completeClose(sendersCur: Long): ChannelSegment<E> { + // Close the linked list for further segment addition, + // obtaining the last segment in the data structure. + val lastSegment = closeLinkedList() + // In the conflated channel implementation (with the DROP_OLDEST + // elements conflation strategy), it is critical to mark all empty + // cells as closed to prevent in-progress `send(e)`-s, which have not + // put their elements yet, completions after this channel is closed. + // Otherwise, it is possible for a `send(e)` to put an element when + // the buffer is already full, while a concurrent receiver may extract + // the oldest element. When the channel is not closed, we can linearize + // this `receive()` before the `send(e)`, but after the channel is closed, + // `send(e)` must fails. Marking all unprocessed cells as `CLOSED` solves the issue. + if (isConflatedDropOldest) { + val lastBufferedCellGlobalIndex = markAllEmptyCellsAsClosed(lastSegment) + if (lastBufferedCellGlobalIndex != -1L) + dropFirstElementUntilTheSpecifiedCellIsInTheBuffer(lastBufferedCellGlobalIndex) + } + // Resume waiting `receive()` requests, + // informing them that the channel is closed. + cancelSuspendedReceiveRequests(lastSegment, sendersCur) + // Return the last segment in the linked list as a result + // of this function; we need it in `completeCancel(..)`. + return lastSegment + } + + /** + * Completes the channel cancellation procedure. + */ + private fun completeCancel(sendersCur: Long) { + // First, ensure that this channel is closed, + // obtaining the last segment in the linked list. + val lastSegment = completeClose(sendersCur) + // Cancel suspended `send(e)` requests and + // remove buffered elements in the reverse order. + removeUnprocessedElements(lastSegment) + } + + /** + * Closes the underlying linked list of segments for further segment addition. + */ + private fun closeLinkedList(): ChannelSegment<E> { + // Choose the last segment. + var lastSegment = bufferEndSegment.value + sendSegment.value.let { if (it.id > lastSegment.id) lastSegment = it } + receiveSegment.value.let { if (it.id > lastSegment.id) lastSegment = it } + // Close the linked list of segment for new segment addition + // and return the last segment in the linked list. + return lastSegment.close() + } + + /** + * This function marks all empty cells, in the `null` and [IN_BUFFER] state, + * as closed. Notably, it processes the cells from right to left, and finishes + * immediately when the processing cell is already covered by `receive()` or + * contains a buffered elements ([BUFFERED] state). + * + * This function returns the global index of the last buffered element, + * or `-1` if this channel does not contain buffered elements. + */ + private fun markAllEmptyCellsAsClosed(lastSegment: ChannelSegment<E>): Long { + // Process the cells in reverse order, from right to left. + var segment = lastSegment + while (true) { + for (index in SEGMENT_SIZE - 1 downTo 0) { + // Is this cell already covered by `receive()`? + val globalIndex = segment.id * SEGMENT_SIZE + index + if (globalIndex < receiversCounter) return -1 + // Process the cell `segment[index]`. + cell_update@ while (true) { + val state = segment.getState(index) + when { + // The cell is empty. + state === null || state === IN_BUFFER -> { + // Inform a possibly upcoming sender that this channel is already closed. + if (segment.casState(index, state, CHANNEL_CLOSED)) { + segment.onSlotCleaned() + break@cell_update + } + } + // The cell stores a buffered element. + state === BUFFERED -> return globalIndex + // Skip this cell if it is not empty and does not store a buffered element. + else -> break@cell_update + } + } + } + // Process the next segment, finishing if the linked list ends. + segment = segment.prev ?: return -1 + } + } + + /** + * Cancels suspended `send(e)` requests and removes buffered elements + * starting from the last cell in the specified [lastSegment] (it must + * be the physical tail of the underlying linked list) and updating + * the cells in reverse order. + */ + private fun removeUnprocessedElements(lastSegment: ChannelSegment<E>) { + // Read the `onUndeliveredElement` lambda at once. In case it + // throws an exception, this exception is handled and stored in + // the variable below. If multiple exceptions are thrown, the first + // one is stored in the variable, while the others are suppressed. + val onUndeliveredElement = onUndeliveredElement + var undeliveredElementException: UndeliveredElementException? = null // first cancel exception, others suppressed + // To perform synchronization correctly, it is critical to + // process the cells in reverse order, from right to left. + // However, according to the API, suspended senders should + // be cancelled in the order of their suspension. Therefore, + // we need to collect all of them and cancel in the reverse + // order after that. + var suspendedSenders = InlineList<Waiter>() + var segment = lastSegment + process_segments@ while (true) { + for (index in SEGMENT_SIZE - 1 downTo 0) { + // Process the cell `segment[index]`. + val globalIndex = segment.id * SEGMENT_SIZE + index + // Update the cell state. + update_cell@ while (true) { + // Read the current state of the cell. + val state = segment.getState(index) + when { + // The cell is already processed by a receiver. + state === DONE_RCV -> break@process_segments + // The cell stores a buffered element. + state === BUFFERED -> { + // Is the cell already covered by a receiver? + if (globalIndex < receiversCounter) break@process_segments + // Update the cell state to `CHANNEL_CLOSED`. + if (segment.casState(index, state, CHANNEL_CLOSED)) { + // If `onUndeliveredElement` lambda is non-null, call it. + if (onUndeliveredElement != null) { + val element = segment.getElement(index) + undeliveredElementException = onUndeliveredElement.callUndeliveredElementCatchingException(element, undeliveredElementException) + } + // Clean the element field and inform the segment + // that the slot is cleaned to avoid memory leaks. + segment.cleanElement(index) + segment.onSlotCleaned() + break@update_cell + } + } + // The cell is empty. + state === IN_BUFFER || state === null -> { + // Update the cell state to `CHANNEL_CLOSED`. + if (segment.casState(index, state, CHANNEL_CLOSED)) { + // Inform the segment that the slot is cleaned to avoid memory leaks. + segment.onSlotCleaned() + break@update_cell + } + } + // The cell stores a suspended waiter. + state is Waiter || state is WaiterEB -> { + // Is the cell already covered by a receiver? + if (globalIndex < receiversCounter) break@process_segments + // Obtain the sender. + val sender: Waiter = if (state is WaiterEB) state.waiter + else state as Waiter + // Update the cell state to `CHANNEL_CLOSED`. + if (segment.casState(index, state, CHANNEL_CLOSED)) { + // If `onUndeliveredElement` lambda is non-null, call it. + if (onUndeliveredElement != null) { + val element = segment.getElement(index) + undeliveredElementException = onUndeliveredElement.callUndeliveredElementCatchingException(element, undeliveredElementException) + } + // Save the sender for further cancellation. + suspendedSenders += sender + // Clean the element field and inform the segment + // that the slot is cleaned to avoid memory leaks. + segment.cleanElement(index) + segment.onSlotCleaned() + break@update_cell + } + } + // A concurrent receiver is resuming a suspended sender. + // As the cell is covered by a receiver, finish immediately. + state === RESUMING_BY_EB || state === RESUMING_BY_RCV -> break@process_segments + // A concurrent `expandBuffer()` is resuming a suspended sender. + // Wait in a spin-loop until the cell state changes. + state === RESUMING_BY_EB -> continue@update_cell + else -> break@update_cell + } + } + } + // Process the previous segment. + segment = segment.prev ?: break + } + // Cancel suspended senders in their order of addition to this channel. + suspendedSenders.forEachReversed { it.resumeSenderOnCancelledChannel() } + // Throw `UndeliveredElementException` at the end if there was one. + undeliveredElementException?.let { throw it } + } + + /** + * Cancels suspended `receive` requests from the end to the beginning, + * also moving empty cells to the `CHANNEL_CLOSED` state. + */ + private fun cancelSuspendedReceiveRequests(lastSegment: ChannelSegment<E>, sendersCounter: Long) { + // To perform synchronization correctly, it is critical to + // extract suspended requests in the reverse order, + // from the end to the beginning. + // However, according to the API, they should be cancelled + // in the order of their suspension. Therefore, we need to + // collect the suspended requests first, cancelling them + // in the reverse order after that. + var suspendedReceivers = InlineList<Waiter>() + var segment: ChannelSegment<E>? = lastSegment + process_segments@ while (segment != null) { + for (index in SEGMENT_SIZE - 1 downTo 0) { + // Is the cell already covered by a sender? Finish immediately in this case. + if (segment.id * SEGMENT_SIZE + index < sendersCounter) break@process_segments + // Try to move the cell state to `CHANNEL_CLOSED`. + cell_update@ while (true) { + val state = segment.getState(index) + when { + state === null || state === IN_BUFFER -> { + if (segment.casState(index, state, CHANNEL_CLOSED)) { + segment.onSlotCleaned() + break@cell_update + } + } + state is WaiterEB -> { + if (segment.casState(index, state, CHANNEL_CLOSED)) { + suspendedReceivers += state.waiter // save for cancellation. + segment.onCancelledRequest(index = index, receiver = true) + break@cell_update + } + } + state is Waiter -> { + if (segment.casState(index, state, CHANNEL_CLOSED)) { + suspendedReceivers += state // save for cancellation. + segment.onCancelledRequest(index = index, receiver = true) + break@cell_update + } + } + else -> break@cell_update // nothing to cancel. + } + } + } + // Process the previous segment. + segment = segment.prev + } + // Cancel the suspended requests in their order of addition to this channel. + suspendedReceivers.forEachReversed { it.resumeReceiverOnClosedChannel() } + } + + /** + * Resumes this receiver because this channel is closed. + * This function does not take any effect if the operation has already been resumed or cancelled. + */ + private fun Waiter.resumeReceiverOnClosedChannel() = resumeWaiterOnClosedChannel(receiver = true) + + /** + * Resumes this sender because this channel is cancelled. + * This function does not take any effect if the operation has already been resumed or cancelled. + */ + private fun Waiter.resumeSenderOnCancelledChannel() = resumeWaiterOnClosedChannel(receiver = false) + + private fun Waiter.resumeWaiterOnClosedChannel(receiver: Boolean) { + when (this) { + is SendBroadcast -> cont.resume(false) + is CancellableContinuation<*> -> resumeWithException(if (receiver) receiveException else sendException) + is ReceiveCatching<*> -> cont.resume(closed(closeCause)) + is BufferedChannel<*>.BufferedChannelIterator -> tryResumeHasNextOnClosedChannel() + is SelectInstance<*> -> trySelect(this@BufferedChannel, CHANNEL_CLOSED) + else -> error("Unexpected waiter: $this") + } + } + + @ExperimentalCoroutinesApi + override val isClosedForSend: Boolean + get() = sendersAndCloseStatus.value.isClosedForSend0 + + private val Long.isClosedForSend0 get() = + isClosed(this, isClosedForReceive = false) + + @ExperimentalCoroutinesApi + override val isClosedForReceive: Boolean + get() = sendersAndCloseStatus.value.isClosedForReceive0 + + private val Long.isClosedForReceive0 get() = + isClosed(this, isClosedForReceive = true) + + private fun isClosed( + sendersAndCloseStatusCur: Long, + isClosedForReceive: Boolean + ) = when (sendersAndCloseStatusCur.sendersCloseStatus) { + // This channel is active and has not been closed. + CLOSE_STATUS_ACTIVE -> false + // The cancellation procedure has been started but + // not linearized yet, so this channel should be + // considered as active. + CLOSE_STATUS_CANCELLATION_STARTED -> false + // This channel has been successfully closed. + // Help to complete the closing procedure to + // guarantee linearizability, and return `true` + // for senders or the flag whether there still + // exist elements to retrieve for receivers. + CLOSE_STATUS_CLOSED -> { + completeClose(sendersAndCloseStatusCur.sendersCounter) + // When `isClosedForReceive` is `false`, always return `true`. + // Otherwise, it is possible that the channel is closed but + // still has elements to retrieve. + if (isClosedForReceive) !hasElements() else true + } + // This channel has been successfully cancelled. + // Help to complete the cancellation procedure to + // guarantee linearizability and return `true`. + CLOSE_STATUS_CANCELLED -> { + completeCancel(sendersAndCloseStatusCur.sendersCounter) + true + } + else -> error("unexpected close status: ${sendersAndCloseStatusCur.sendersCloseStatus}") + } + + @ExperimentalCoroutinesApi + override val isEmpty: Boolean get() { + // This function should return `false` if + // this channel is closed for `receive`. + if (isClosedForReceive) return false + // Does this channel has elements to retrieve? + if (hasElements()) return false + // This channel does not have elements to retrieve; + // Check that it is still not closed for `receive`. + return !isClosedForReceive + } + + /** + * Checks whether this channel contains elements to retrieve. + * Unfortunately, simply comparing the counters is insufficient, + * as some cells can be in the `INTERRUPTED` state due to cancellation. + * This function tries to find the first "alive" element, + * updating the `receivers` counter to skip empty cells. + * + * The implementation is similar to `receive()`. + */ + internal fun hasElements(): Boolean { + while (true) { + // Read the segment before obtaining the `receivers` counter value. + var segment = receiveSegment.value + // Obtains the `receivers` and `senders` counter values. + val r = receiversCounter + val s = sendersCounter + // Is there a chance that this channel has elements? + if (s <= r) return false // no elements + // The `r`-th cell is covered by a sender; check whether it contains an element. + // First, try to find the required segment if the initially + // obtained segment (in the beginning of this function) has lower id. + val id = r / SEGMENT_SIZE + if (segment.id != id) { + // Try to find the required segment. + segment = findSegmentReceive(id, segment) ?: + // The required segment has not been found. Either it has already + // been removed, or the underlying linked list is already closed + // for segment additions. In the latter case, the channel is closed + // and does not contain elements, so this operation returns `false`. + // Otherwise, if the required segment is removed, the operation restarts. + if (receiveSegment.value.id < id) return false else continue + } + segment.cleanPrev() // all the previous segments are no longer needed. + // Does the `r`-th cell contain waiting sender or buffered element? + val i = (r % SEGMENT_SIZE).toInt() + if (isCellNonEmpty(segment, i, r)) return true + // The cell is empty. Update `receivers` counter and try again. + receivers.compareAndSet(r, r + 1) // if this CAS fails, the counter has already been updated. + } + } + + /** + * Checks whether this cell contains a buffered element or a waiting sender, + * returning `true` in this case. Otherwise, if this cell is empty + * (due to waiter cancellation, cell poisoning, or channel closing), + * this function returns `false`. + * + * Notably, this function must be called only if the cell is covered by a sender. + */ + private fun isCellNonEmpty( + segment: ChannelSegment<E>, + index: Int, + globalIndex: Long + ): Boolean { + // The logic is similar to `updateCellReceive` with the only difference + // that this function neither changes the cell state nor retrieves the element. + while (true) { + // Read the current cell state. + val state = segment.getState(index) + when { + // The cell is empty but a sender is coming. + state === null || state === IN_BUFFER -> { + // Poison the cell to ensure correctness. + if (segment.casState(index, state, POISONED)) { + // When the cell becomes poisoned, it is essentially + // the same as storing an already cancelled receiver. + // Thus, the `expandBuffer()` procedure should be invoked. + expandBuffer() + return false + } + } + // The cell stores a buffered element. + state === BUFFERED -> return true + // The cell stores an interrupted sender. + state === INTERRUPTED_SEND -> return false + // This channel is already closed. + state === CHANNEL_CLOSED -> return false + // The cell is already processed + // by a concurrent receiver. + state === DONE_RCV -> return false + // The cell is already poisoned + // by a concurrent receiver. + state === POISONED -> return false + // A concurrent `expandBuffer()` is resuming + // a suspended sender. This function is eligible + // to linearize before the buffer expansion procedure. + state === RESUMING_BY_EB -> return true + // A concurrent receiver is resuming + // a suspended sender. The element + // is no longer available for retrieval. + state === RESUMING_BY_RCV -> return false + // The cell stores a suspended request. + // However, it is possible that this request + // is receiver if the cell is covered by both + // send and receive operations. + // In case the cell is already covered by + // a receiver, the element is no longer + // available for retrieval, and this function + // return `false`. Otherwise, it is guaranteed + // that the suspended request is sender, so + // this function returns `true`. + else -> return globalIndex == receiversCounter + } + } + } + + // ####################### + // # Segments Management # + // ####################### + + /** + * Finds the segment with the specified [id] starting by the [startFrom] + * segment and following the [ChannelSegment.next] references. In case + * the required segment has not been created yet, this function attempts + * to add it to the underlying linked list. Finally, it updates [sendSegment] + * to the found segment if its [ChannelSegment.id] is greater than the one + * of the already stored segment. + * + * In case the requested segment is already removed, or if it should be allocated + * but the linked list structure is closed for new segments addition, this function + * returns `null`. The implementation also efficiently skips a sequence of removed + * segments, updating the counter value in [sendersAndCloseStatus] correspondingly. + */ + private fun findSegmentSend(id: Long, startFrom: ChannelSegment<E>): ChannelSegment<E>? { + return sendSegment.findSegmentAndMoveForward(id, startFrom, createSegmentFunction()).let { + if (it.isClosed) { + // The required segment has not been found and new segments + // cannot be added, as the linked listed in already added. + // This channel is already closed or cancelled; help to complete + // the closing or cancellation procedure. + completeCloseOrCancel() + // Clean the `prev` reference of the provided segment + // if all the previous cells are already covered by senders. + // It is important to clean the `prev` reference only in + // this case, as the closing/cancellation procedure may + // need correct value to traverse the linked list from right to left. + if (startFrom.id * SEGMENT_SIZE < receiversCounter) startFrom.cleanPrev() + // As the required segment is not found and cannot be allocated, return `null`. + null + } else { + // Get the found segment. + val segment = it.segment + // Is the required segment removed? + if (segment.id > id) { + // The required segment has been removed; `segment` is the first + // segment with `id` not lower than the required one. + // Skip the sequence of removed cells in O(1). + updateSendersCounterIfLower(segment.id * SEGMENT_SIZE) + // Clean the `prev` reference of the provided segment + // if all the previous cells are already covered by senders. + // It is important to clean the `prev` reference only in + // this case, as the closing/cancellation procedure may + // need correct value to traverse the linked list from right to left. + if (segment.id * SEGMENT_SIZE < receiversCounter) segment.cleanPrev() + // As the required segment is not found and cannot be allocated, return `null`. + null + } else { + assert { segment.id == id } + // The required segment has been found; return it! + segment + } + } + } + } + + /** + * Finds the segment with the specified [id] starting by the [startFrom] + * segment and following the [ChannelSegment.next] references. In case + * the required segment has not been created yet, this function attempts + * to add it to the underlying linked list. Finally, it updates [receiveSegment] + * to the found segment if its [ChannelSegment.id] is greater than the one + * of the already stored segment. + * + * In case the requested segment is already removed, or if it should be allocated + * but the linked list structure is closed for new segments addition, this function + * returns `null`. The implementation also efficiently skips a sequence of removed + * segments, updating the [receivers] counter correspondingly. + */ + private fun findSegmentReceive(id: Long, startFrom: ChannelSegment<E>): ChannelSegment<E>? = + receiveSegment.findSegmentAndMoveForward(id, startFrom, createSegmentFunction()).let { + if (it.isClosed) { + // The required segment has not been found and new segments + // cannot be added, as the linked listed in already added. + // This channel is already closed or cancelled; help to complete + // the closing or cancellation procedure. + completeCloseOrCancel() + // Clean the `prev` reference of the provided segment + // if all the previous cells are already covered by senders. + // It is important to clean the `prev` reference only in + // this case, as the closing/cancellation procedure may + // need correct value to traverse the linked list from right to left. + if (startFrom.id * SEGMENT_SIZE < sendersCounter) startFrom.cleanPrev() + // As the required segment is not found and cannot be allocated, return `null`. + null + } else { + // Get the found segment. + val segment = it.segment + // Advance the `bufferEnd` segment if required. + if (!isRendezvousOrUnlimited && id <= bufferEndCounter / SEGMENT_SIZE) { + bufferEndSegment.moveForward(segment) + } + // Is the required segment removed? + if (segment.id > id) { + // The required segment has been removed; `segment` is the first + // segment with `id` not lower than the required one. + // Skip the sequence of removed cells in O(1). + updateReceiversCounterIfLower(segment.id * SEGMENT_SIZE) + // Clean the `prev` reference of the provided segment + // if all the previous cells are already covered by senders. + // It is important to clean the `prev` reference only in + // this case, as the closing/cancellation procedure may + // need correct value to traverse the linked list from right to left. + if (segment.id * SEGMENT_SIZE < sendersCounter) segment.cleanPrev() + // As the required segment is already removed, return `null`. + null + } else { + assert { segment.id == id } + // The required segment has been found; return it! + segment + } + } + } + + /** + * Importantly, when this function does not find the requested segment, + * it always updates the number of completed `expandBuffer()` attempts. + */ + private fun findSegmentBufferEnd(id: Long, startFrom: ChannelSegment<E>, currentBufferEndCounter: Long): ChannelSegment<E>? = + bufferEndSegment.findSegmentAndMoveForward(id, startFrom, createSegmentFunction()).let { + if (it.isClosed) { + // The required segment has not been found and new segments + // cannot be added, as the linked listed in already added. + // This channel is already closed or cancelled; help to complete + // the closing or cancellation procedure. + completeCloseOrCancel() + // Update `bufferEndSegment` to the last segment + // in the linked list to avoid memory leaks. + moveSegmentBufferEndToSpecifiedOrLast(id, startFrom) + // When this function does not find the requested segment, + // it should update the number of completed `expandBuffer()` attempts. + incCompletedExpandBufferAttempts() + null + } else { + // Get the found segment. + val segment = it.segment + // Is the required segment removed? + if (segment.id > id) { + // The required segment has been removed; `segment` is the first segment + // with `id` not lower than the required one. + // Try to skip the sequence of removed cells in O(1) by increasing the `bufferEnd` counter. + // Importantly, when this function does not find the requested segment, + // it should update the number of completed `expandBuffer()` attempts. + if (bufferEnd.compareAndSet(currentBufferEndCounter + 1, segment.id * SEGMENT_SIZE)) { + incCompletedExpandBufferAttempts(segment.id * SEGMENT_SIZE - currentBufferEndCounter) + } else { + incCompletedExpandBufferAttempts() + } + // As the required segment is already removed, return `null`. + null + } else { + assert { segment.id == id } + // The required segment has been found; return it! + segment + } + } + } + + /** + * Updates [bufferEndSegment] to the one with the specified [id] or + * to the last existing segment, if the required segment is not yet created. + * + * Unlike [findSegmentBufferEnd], this function does not allocate new segments. + */ + private fun moveSegmentBufferEndToSpecifiedOrLast(id: Long, startFrom: ChannelSegment<E>) { + // Start searching the required segment from the specified one. + var segment: ChannelSegment<E> = startFrom + while (segment.id < id) { + segment = segment.next ?: break + } + // Skip all removed segments and try to update `bufferEndSegment` + // to the first non-removed one. This part should succeed eventually, + // as the tail segment is never removed. + while (true) { + while (segment.isRemoved) { + segment = segment.next ?: break + } + // Try to update `bufferEndSegment`. On failure, + // the found segment is already removed, so it + // should be skipped. + if (bufferEndSegment.moveForward(segment)) return + } + } + + /** + * Updates the `senders` counter if its value + * is lower that the specified one. + * + * Senders use this function to efficiently skip + * a sequence of cancelled receivers. + */ + private fun updateSendersCounterIfLower(value: Long): Unit = + sendersAndCloseStatus.loop { cur -> + val curCounter = cur.sendersCounter + if (curCounter >= value) return + val update = constructSendersAndCloseStatus(curCounter, cur.sendersCloseStatus) + if (sendersAndCloseStatus.compareAndSet(cur, update)) return + } + + /** + * Updates the `receivers` counter if its value + * is lower that the specified one. + * + * Receivers use this function to efficiently skip + * a sequence of cancelled senders. + */ + private fun updateReceiversCounterIfLower(value: Long): Unit = + receivers.loop { cur -> + if (cur >= value) return + if (receivers.compareAndSet(cur, value)) return + } + + // ################### + // # Debug Functions # + // ################### + + @Suppress("ConvertTwoComparisonsToRangeCheck") + override fun toString(): String { + val sb = StringBuilder() + // Append the close status + when (sendersAndCloseStatus.value.sendersCloseStatus) { + CLOSE_STATUS_CLOSED -> sb.append("closed,") + CLOSE_STATUS_CANCELLED -> sb.append("cancelled,") + } + // Append the buffer capacity + sb.append("capacity=$capacity,") + // Append the data + sb.append("data=[") + val firstSegment = listOf(receiveSegment.value, sendSegment.value, bufferEndSegment.value) + .filter { it !== NULL_SEGMENT } + .minBy { it.id } + val r = receiversCounter + val s = sendersCounter + var segment = firstSegment + append_elements@ while (true) { + process_cell@ for (i in 0 until SEGMENT_SIZE) { + val globalCellIndex = segment.id * SEGMENT_SIZE + i + if (globalCellIndex >= s && globalCellIndex >= r) break@append_elements + val cellState = segment.getState(i) + val element = segment.getElement(i) + val cellStateString = when (cellState) { + is CancellableContinuation<*> -> { + when { + globalCellIndex < r && globalCellIndex >= s -> "receive" + globalCellIndex < s && globalCellIndex >= r -> "send" + else -> "cont" + } + } + is SelectInstance<*> -> { + when { + globalCellIndex < r && globalCellIndex >= s -> "onReceive" + globalCellIndex < s && globalCellIndex >= r -> "onSend" + else -> "select" + } + } + is ReceiveCatching<*> -> "receiveCatching" + is SendBroadcast -> "sendBroadcast" + is WaiterEB -> "EB($cellState)" + RESUMING_BY_RCV, RESUMING_BY_EB -> "resuming_sender" + null, IN_BUFFER, DONE_RCV, POISONED, INTERRUPTED_RCV, INTERRUPTED_SEND, CHANNEL_CLOSED -> continue@process_cell + else -> cellState.toString() // leave it just in case something is missed. + } + if (element != null) { + sb.append("($cellStateString,$element),") + } else { + sb.append("$cellStateString,") + } + } + // Process the next segment if exists. + segment = segment.next ?: break + } + if (sb.last() == ',') sb.deleteAt(sb.length - 1) + sb.append("]") + // The string representation is constructed. + return sb.toString() + } + + // Returns a debug representation of this channel, + // which is actively used in Lincheck tests. + internal fun toStringDebug(): String { + val sb = StringBuilder() + // Append the counter values and the close status + sb.append("S=${sendersCounter},R=${receiversCounter},B=${bufferEndCounter},B'=${completedExpandBuffersAndPauseFlag.value},C=${sendersAndCloseStatus.value.sendersCloseStatus},") + when (sendersAndCloseStatus.value.sendersCloseStatus) { + CLOSE_STATUS_CANCELLATION_STARTED -> sb.append("CANCELLATION_STARTED,") + CLOSE_STATUS_CLOSED -> sb.append("CLOSED,") + CLOSE_STATUS_CANCELLED -> sb.append("CANCELLED,") + } + // Append the segment references + sb.append("SEND_SEGM=${sendSegment.value.hexAddress},RCV_SEGM=${receiveSegment.value.hexAddress}") + if (!isRendezvousOrUnlimited) sb.append(",EB_SEGM=${bufferEndSegment.value.hexAddress}") + sb.append(" ") // add some space + // Append the linked list of segments. + val firstSegment = listOf(receiveSegment.value, sendSegment.value, bufferEndSegment.value) + .filter { it !== NULL_SEGMENT } + .minBy { it.id } + var segment = firstSegment + while (true) { + sb.append("${segment.hexAddress}=[${if (segment.isRemoved) "*" else ""}${segment.id},prev=${segment.prev?.hexAddress},") + repeat(SEGMENT_SIZE) { i -> + val cellState = segment.getState(i) + val element = segment.getElement(i) + val cellStateString = when (cellState) { + is CancellableContinuation<*> -> "cont" + is SelectInstance<*> -> "select" + is ReceiveCatching<*> -> "receiveCatching" + is SendBroadcast -> "send(broadcast)" + is WaiterEB -> "EB($cellState)" + else -> cellState.toString() + } + sb.append("[$i]=($cellStateString,$element),") + } + sb.append("next=${segment.next?.hexAddress}] ") + // Process the next segment if exists. + segment = segment.next ?: break + } + // The string representation of this channel is now constructed! + return sb.toString() + } + + + // This is an internal methods for tests. + fun checkSegmentStructureInvariants() { + if (isRendezvousOrUnlimited) { + check(bufferEndSegment.value === NULL_SEGMENT) { + "bufferEndSegment must be NULL_SEGMENT for rendezvous and unlimited channels; they do not manipulate it.\n" + + "Channel state: $this" + } + } else { + check(receiveSegment.value.id <= bufferEndSegment.value.id) { + "bufferEndSegment should not have lower id than receiveSegment.\n" + + "Channel state: $this" + } + } + val firstSegment = listOf(receiveSegment.value, sendSegment.value, bufferEndSegment.value) + .filter { it !== NULL_SEGMENT } + .minBy { it.id } + check(firstSegment.prev == null) { + "All processed segments should be unreachable from the data structure, but the `prev` link of the leftmost segment is non-null.\n" + + "Channel state: $this" + } + // Check that the doubly-linked list of segments does not + // contain full-of-cancelled-cells segments. + var segment = firstSegment + while (segment.next != null) { + // Note that the `prev` reference can be `null` if this channel is closed. + check(segment.next!!.prev == null || segment.next!!.prev === segment) { + "The `segment.next.prev === segment` invariant is violated.\n" + + "Channel state: $this" + } + // Count the number of closed/interrupted cells + // and check that all cells are in expected states. + var interruptedOrClosedCells = 0 + for (i in 0 until SEGMENT_SIZE) { + when (val state = segment.getState(i)) { + BUFFERED -> {} // The cell stores a buffered element. + is Waiter -> {} // The cell stores a suspended request. + INTERRUPTED_RCV, INTERRUPTED_SEND, CHANNEL_CLOSED -> { + // The cell stored an interrupted request or indicates + // that this channel is already closed. + // Check that the element slot is cleaned and increment + // the number of cells in closed/interrupted state. + check(segment.getElement(i) == null) + interruptedOrClosedCells++ + } + POISONED, DONE_RCV -> { + // The cell is successfully processed or poisoned. + // Check that the element slot is cleaned. + check(segment.getElement(i) == null) + } + // Other states are illegal after all running operations finish. + else -> error("Unexpected segment cell state: $state.\nChannel state: $this") + } + } + // Is this segment full of cancelled/closed cells? + // If so, this segment should be removed from the + // linked list if nether `receiveSegment`, nor + // `sendSegment`, nor `bufferEndSegment` reference it. + if (interruptedOrClosedCells == SEGMENT_SIZE) { + check(segment === receiveSegment.value || segment === sendSegment.value || segment === bufferEndSegment.value) { + "Logically removed segment is reachable.\nChannel state: $this" + } + } + // Process the next segment. + segment = segment.next!! + } + } +} + +/** + * The channel is represented as a list of segments, which simulates an infinite array. + * Each segment has its own [id], which increase from the beginning. These [id]s help + * to update [BufferedChannel.sendSegment], [BufferedChannel.receiveSegment], + * and [BufferedChannel.bufferEndSegment] correctly. + */ +internal class ChannelSegment<E>(id: Long, prev: ChannelSegment<E>?, channel: BufferedChannel<E>?, pointers: Int) : Segment<ChannelSegment<E>>(id, prev, pointers) { + private val _channel: BufferedChannel<E>? = channel + val channel get() = _channel!! // always non-null except for `NULL_SEGMENT` + + private val data = atomicArrayOfNulls<Any?>(SEGMENT_SIZE * 2) // 2 registers per slot: state + element + override val numberOfSlots: Int get() = SEGMENT_SIZE + + // ######################################## + // # Manipulation with the Element Fields # + // ######################################## + + internal fun storeElement(index: Int, element: E) { + setElementLazy(index, element) + } + + @Suppress("UNCHECKED_CAST") + internal fun getElement(index: Int) = data[index * 2].value as E + + internal fun retrieveElement(index: Int): E = getElement(index).also { cleanElement(index) } + + internal fun cleanElement(index: Int) { + setElementLazy(index, null) + } + + private fun setElementLazy(index: Int, value: Any?) { + data[index * 2].lazySet(value) + } + + // ###################################### + // # Manipulation with the State Fields # + // ###################################### + + internal fun getState(index: Int): Any? = data[index * 2 + 1].value + + internal fun setState(index: Int, value: Any?) { + data[index * 2 + 1].value = value + } + + internal fun casState(index: Int, from: Any?, to: Any?) = data[index * 2 + 1].compareAndSet(from, to) + + internal fun getAndSetState(index: Int, update: Any?) = data[index * 2 + 1].getAndSet(update) + + + // ######################## + // # Cancellation Support # + // ######################## + + override fun onCancellation(index: Int, cause: Throwable?, context: CoroutineContext) { + // To distinguish cancelled senders and receivers, senders equip the index value with + // an additional marker, adding `SEGMENT_SIZE` to the value. + val isSender = index >= SEGMENT_SIZE + // Unwrap the index. + @Suppress("NAME_SHADOWING") val index = if (isSender) index - SEGMENT_SIZE else index + // Read the element, which may be needed further to call `onUndeliveredElement`. + val element = getElement(index) + // Update the cell state. + while (true) { + // CAS-loop + // Read the current state of the cell. + val cur = getState(index) + when { + // The cell stores a waiter. + cur is Waiter || cur is WaiterEB -> { + // The cancelled request is either send or receive. + // Update the cell state correspondingly. + val update = if (isSender) INTERRUPTED_SEND else INTERRUPTED_RCV + if (casState(index, cur, update)) { + // The waiter has been successfully cancelled. + // Clean the element slot and invoke `onSlotCleaned()`, + // which may cause deleting the whole segment from the linked list. + // In case the cancelled request is receiver, it is critical to ensure + // that the `expandBuffer()` attempt that processes this cell is completed, + // so `onCancelledRequest(..)` waits for its completion before invoking `onSlotCleaned()`. + cleanElement(index) + onCancelledRequest(index, !isSender) + // Call `onUndeliveredElement` if needed. + if (isSender) { + channel.onUndeliveredElement?.callUndeliveredElement(element, context) + } + return + } + } + // The cell already indicates that the operation is cancelled. + cur === INTERRUPTED_SEND || cur === INTERRUPTED_RCV -> { + // Clean the element slot to avoid memory leaks, + // invoke `onUndeliveredElement` if needed, and finish + cleanElement(index) + // Call `onUndeliveredElement` if needed. + if (isSender) { + channel.onUndeliveredElement?.callUndeliveredElement(element, context) + } + return + } + // An opposite operation is resuming this request; + // wait until the cell state updates. + // It is possible that an opposite operation has already + // resumed this request, which will result in updating + // the cell state to `DONE_RCV` or `BUFFERED`, while the + // current cancellation is caused by prompt cancellation. + cur === RESUMING_BY_EB || cur === RESUMING_BY_RCV -> continue + // This request was successfully resumed, so this cancellation + // is caused by the prompt cancellation feature and should be ignored. + cur === DONE_RCV || cur === BUFFERED -> return + // The cell state indicates that the channel is closed; + // this cancellation should be ignored. + cur === CHANNEL_CLOSED -> return + else -> error("unexpected state: $cur") + } + } + } + + /** + * Invokes `onSlotCleaned()` preceded by a `waitExpandBufferCompletion(..)` call + * in case the cancelled request is receiver. + */ + fun onCancelledRequest(index: Int, receiver: Boolean) { + if (receiver) channel.waitExpandBufferCompletion(id * SEGMENT_SIZE + index) + onSlotCleaned() + } +} + +// WA for atomicfu + JVM_IR compiler bug that lead to SMAP-related compiler crashes: KT-55983 +internal fun <E> createSegmentFunction(): KFunction2<Long, ChannelSegment<E>, ChannelSegment<E>> = ::createSegment + +private fun <E> createSegment(id: Long, prev: ChannelSegment<E>) = ChannelSegment( + id = id, + prev = prev, + channel = prev.channel, + pointers = 0 +) +private val NULL_SEGMENT = ChannelSegment<Any?>(id = -1, prev = null, channel = null, pointers = 0) + +/** + * Number of cells in each segment. + */ +@JvmField +internal val SEGMENT_SIZE = systemProp("kotlinx.coroutines.bufferedChannel.segmentSize", 32) + +/** + * Number of iterations to wait in [BufferedChannel.waitExpandBufferCompletion] until the numbers of started and completed + * [BufferedChannel.expandBuffer] calls coincide. When the limit is reached, [BufferedChannel.waitExpandBufferCompletion] + * blocks further [BufferedChannel.expandBuffer]-s to avoid starvation. + */ +private val EXPAND_BUFFER_COMPLETION_WAIT_ITERATIONS = systemProp("kotlinx.coroutines.bufferedChannel.expandBufferCompletionWaitIterations", 10_000) + +/** + * Tries to resume this continuation with the specified + * value. Returns `true` on success and `false` on failure. + */ +private fun <T> CancellableContinuation<T>.tryResume0( + value: T, + onCancellation: ((cause: Throwable) -> Unit)? = null +): Boolean = + tryResume(value, null, onCancellation).let { token -> + if (token != null) { + completeResume(token) + true + } else false + } + +/* + If the channel is rendezvous or unlimited, the `bufferEnd` counter + should be initialized with the corresponding value below and never change. + In this case, the `expandBuffer(..)` operation does nothing. + */ +private const val BUFFER_END_RENDEZVOUS = 0L // no buffer +private const val BUFFER_END_UNLIMITED = Long.MAX_VALUE // infinite buffer +private fun initialBufferEnd(capacity: Int): Long = when (capacity) { + Channel.RENDEZVOUS -> BUFFER_END_RENDEZVOUS + Channel.UNLIMITED -> BUFFER_END_UNLIMITED + else -> capacity.toLong() +} + +/* + Cell states. The initial "empty" state is represented with `null`, + and suspended operations are represented with [Waiter] instances. + */ + +// The cell stores a buffered element. +@JvmField +internal val BUFFERED = Symbol("BUFFERED") +// Concurrent `expandBuffer(..)` can inform the +// upcoming sender that it should buffer the element. +private val IN_BUFFER = Symbol("SHOULD_BUFFER") +// Indicates that a receiver (RCV suffix) is resuming +// the suspended sender; after that, it should update +// the state to either `DONE_RCV` (on success) or +// `INTERRUPTED_SEND` (on failure). +private val RESUMING_BY_RCV = Symbol("S_RESUMING_BY_RCV") +// Indicates that `expandBuffer(..)` (RCV suffix) is resuming +// the suspended sender; after that, it should update +// the state to either `BUFFERED` (on success) or +// `INTERRUPTED_SEND` (on failure). +private val RESUMING_BY_EB = Symbol("RESUMING_BY_EB") +// When a receiver comes to the cell already covered by +// a sender (according to the counters), but the cell +// is still in `EMPTY` or `IN_BUFFER` state, it breaks +// the cell by changing its state to `POISONED`. +private val POISONED = Symbol("POISONED") +// When the element is successfully transferred +// to a receiver, the cell changes to `DONE_RCV`. +private val DONE_RCV = Symbol("DONE_RCV") +// Cancelled sender. +private val INTERRUPTED_SEND = Symbol("INTERRUPTED_SEND") +// Cancelled receiver. +private val INTERRUPTED_RCV = Symbol("INTERRUPTED_RCV") +// Indicates that the channel is closed. +internal val CHANNEL_CLOSED = Symbol("CHANNEL_CLOSED") +// When the cell is already covered by both sender and +// receiver (`sender` and `receivers` counters are greater +// than the cell number), the `expandBuffer(..)` procedure +// cannot distinguish which kind of operation is stored +// in the cell. Thus, it wraps the waiter with this descriptor, +// informing the possibly upcoming receiver that it should +// complete the `expandBuffer(..)` procedure if the waiter stored +// in the cell is sender. In turn, senders ignore this information. +private class WaiterEB(@JvmField val waiter: Waiter) { + override fun toString() = "WaiterEB($waiter)" +} + + + +/** + * To distinguish suspended [BufferedChannel.receive] and + * [BufferedChannel.receiveCatching] operations, the latter + * uses this wrapper for its continuation. + */ +private class ReceiveCatching<E>( + @JvmField val cont: CancellableContinuationImpl<ChannelResult<E>> +) : Waiter by cont + +/* + Internal results for [BufferedChannel.updateCellReceive]. + On successful rendezvous with waiting sender or + buffered element retrieval, the corresponding element + is returned as result of [BufferedChannel.updateCellReceive]. + */ +private val SUSPEND = Symbol("SUSPEND") +private val SUSPEND_NO_WAITER = Symbol("SUSPEND_NO_WAITER") +private val FAILED = Symbol("FAILED") + +/* + Internal results for [BufferedChannel.updateCellSend] + */ +private const val RESULT_RENDEZVOUS = 0 +private const val RESULT_BUFFERED = 1 +private const val RESULT_SUSPEND = 2 +private const val RESULT_SUSPEND_NO_WAITER = 3 +private const val RESULT_CLOSED = 4 +private const val RESULT_FAILED = 5 + +/** + * Special value for [BufferedChannel.BufferedChannelIterator.receiveResult] + * that indicates the absence of pre-received result. + */ +private val NO_RECEIVE_RESULT = Symbol("NO_RECEIVE_RESULT") + +/* + As [BufferedChannel.invokeOnClose] can be invoked concurrently + with channel closing, we have to synchronize them. These two + markers help with the synchronization. + */ +private val CLOSE_HANDLER_CLOSED = Symbol("CLOSE_HANDLER_CLOSED") +private val CLOSE_HANDLER_INVOKED = Symbol("CLOSE_HANDLER_INVOKED") + +/** + * Specifies the absence of closing cause, stored in [BufferedChannel._closeCause]. + * When the channel is closed or cancelled without exception, this [NO_CLOSE_CAUSE] + * marker should be replaced with `null`. + */ +private val NO_CLOSE_CAUSE = Symbol("NO_CLOSE_CAUSE") + +/* + The channel close statuses. The transition scheme is the following: + +--------+ +----------------------+ +-----------+ + | ACTIVE |-->| CANCELLATION_STARTED |-->| CANCELLED | + +--------+ +----------------------+ +-----------+ + | ^ + | +--------+ | + +------------>| CLOSED |------------------+ + +--------+ + We need `CANCELLATION_STARTED` to synchronize + concurrent closing and cancellation. + */ +private const val CLOSE_STATUS_ACTIVE = 0 +private const val CLOSE_STATUS_CANCELLATION_STARTED = 1 +private const val CLOSE_STATUS_CLOSED = 2 +private const val CLOSE_STATUS_CANCELLED = 3 + +/* + The `senders` counter and the channel close status + are stored in a single 64-bit register to save the space + and reduce the number of reads in sending operations. + The code below encapsulates the required bit arithmetics. + */ +private const val SENDERS_CLOSE_STATUS_SHIFT = 60 +private const val SENDERS_COUNTER_MASK = (1L shl SENDERS_CLOSE_STATUS_SHIFT) - 1 +private inline val Long.sendersCounter get() = this and SENDERS_COUNTER_MASK +private inline val Long.sendersCloseStatus: Int get() = (this shr SENDERS_CLOSE_STATUS_SHIFT).toInt() +private fun constructSendersAndCloseStatus(counter: Long, closeStatus: Int): Long = + (closeStatus.toLong() shl SENDERS_CLOSE_STATUS_SHIFT) + counter + +/* + The `completedExpandBuffersAndPauseFlag` 64-bit counter contains + the number of completed `expandBuffer()` attempts along with a special + flag that pauses progress to avoid starvation in `waitExpandBufferCompletion(..)`. + The code below encapsulates the required bit arithmetics. + */ +private const val EB_COMPLETED_PAUSE_EXPAND_BUFFERS_BIT = 1L shl 62 +private const val EB_COMPLETED_COUNTER_MASK = EB_COMPLETED_PAUSE_EXPAND_BUFFERS_BIT - 1 +private inline val Long.ebCompletedCounter get() = this and EB_COMPLETED_COUNTER_MASK +private inline val Long.ebPauseExpandBuffers: Boolean get() = (this and EB_COMPLETED_PAUSE_EXPAND_BUFFERS_BIT) != 0L +private fun constructEBCompletedAndPauseFlag(counter: Long, pauseEB: Boolean): Long = + (if (pauseEB) EB_COMPLETED_PAUSE_EXPAND_BUFFERS_BIT else 0) + counter diff --git a/kotlinx-coroutines-core/common/src/channels/Channel.kt b/kotlinx-coroutines-core/common/src/channels/Channel.kt index 5ad79fdc..b14e61bb 100644 --- a/kotlinx-coroutines-core/common/src/channels/Channel.kt +++ b/kotlinx-coroutines-core/common/src/channels/Channel.kt @@ -23,12 +23,17 @@ import kotlin.jvm.* */ public interface SendChannel<in E> { /** - * Returns `true` if this channel was closed by an invocation of [close]. This means that - * calling [send] will result in an exception. + * Returns `true` if this channel was closed by an invocation of [close] or its receiving side was [cancelled][ReceiveChannel.cancel]. + * This means that calling [send] will result in an exception. * - * **Note: This is an experimental api.** This property may change its semantics and/or name in the future. + * Note that if this property returns `false`, it does not guarantee that consecutive call to [send] will succeed, as the + * channel can be concurrently closed right after the check. For such scenarios, it is recommended to use [trySend] instead. + * + * @see SendChannel.trySend + * @see SendChannel.close + * @see ReceiveChannel.cancel */ - @ExperimentalCoroutinesApi + @DelicateCoroutinesApi public val isClosedForSend: Boolean /** @@ -100,33 +105,40 @@ public interface SendChannel<in E> { * If the channel is closed already, the handler is invoked immediately. * * The meaning of `cause` that is passed to the handler: - * * `null` if the channel was closed or cancelled without the corresponding argument - * * the cause of `close` or `cancel` otherwise. + * - `null` if the channel was closed normally without the corresponding argument. + * - Instance of [CancellationException] if the channel was cancelled normally without the corresponding argument. + * - The cause of `close` or `cancel` otherwise. * - * Example of usage (exception handling is omitted): + * ### Execution context and exception safety * + * The [handler] is executed as part of the closing or cancelling operation, and only after the channel reaches its final state. + * This means that if the handler throws an exception or hangs, the channel will still be successfully closed or cancelled. + * Unhandled exceptions from [handler] are propagated to the closing or cancelling operation's caller. + * + * Example of usage: * ``` - * val events = Channel(UNLIMITED) + * val events = Channel<Event>(UNLIMITED) * callbackBasedApi.registerCallback { event -> * events.trySend(event) + * .onClosed { /* channel is already closed, but the callback hasn't stopped yet */ } * } * - * val uiUpdater = launch(Dispatchers.Main, parent = UILifecycle) { - * events.consume {} - * events.cancel() + * val uiUpdater = uiScope.launch(Dispatchers.Main) { + * events.consume { /* handle events */ } * } - * + * // Stop the callback after the channel is closed or cancelled * events.invokeOnClose { callbackBasedApi.stop() } * ``` * - * **Note: This is an experimental api.** This function may change its semantics, parameters or return type in the future. + * **Stability note.** This function constitutes a stable API surface, with the only exception being + * that an [IllegalStateException] is thrown when multiple handlers are registered. + * This restriction could be lifted in the future. * - * @throws UnsupportedOperationException if the underlying channel doesn't support [invokeOnClose]. + * @throws UnsupportedOperationException if the underlying channel does not support [invokeOnClose]. * Implementation note: currently, [invokeOnClose] is unsupported only by Rx-like integrations * * @throws IllegalStateException if another handler was already registered */ - @ExperimentalCoroutinesApi public fun invokeOnClose(handler: (cause: Throwable?) -> Unit) /** @@ -160,7 +172,7 @@ public interface SendChannel<in E> { level = DeprecationLevel.ERROR, message = "Deprecated in the favour of 'trySend' method", replaceWith = ReplaceWith("trySend(element).isSuccess") - ) // Warning since 1.5.0, error since 1.6.0 + ) // Warning since 1.5.0, error since 1.6.0, not hidden until 1.8+ because API is quite widespread public fun offer(element: E): Boolean { val result = trySend(element) if (result.isSuccess) return true @@ -174,14 +186,20 @@ public interface SendChannel<in E> { public interface ReceiveChannel<out E> { /** * Returns `true` if this channel was closed by invocation of [close][SendChannel.close] on the [SendChannel] - * side and all previously sent items were already received. This means that calling [receive] - * will result in a [ClosedReceiveChannelException]. If the channel was closed because of an exception, it - * is considered closed, too, but is called a _failed_ channel. All suspending attempts to receive - * an element from a failed channel throw the original [close][SendChannel.close] cause exception. + * side and all previously sent items were already received, or if the receiving side was [cancelled][ReceiveChannel.cancel]. + * + * This means that calling [receive] will result in a [ClosedReceiveChannelException] or a corresponding cancellation cause. + * If the channel was closed because of an exception, it is considered closed, too, but is called a _failed_ channel. + * All suspending attempts to receive an element from a failed channel throw the original [close][SendChannel.close] cause exception. * - * **Note: This is an experimental api.** This property may change its semantics and/or name in the future. + * Note that if this property returns `false`, it does not guarantee that consecutive call to [receive] will succeed, as the + * channel can be concurrently closed right after the check. For such scenarios, it is recommended to use [receiveCatching] instead. + * + * @see ReceiveChannel.receiveCatching + * @see ReceiveChannel.cancel + * @see SendChannel.close */ - @ExperimentalCoroutinesApi + @DelicateCoroutinesApi public val isClosedForReceive: Boolean /** @@ -318,7 +336,7 @@ public interface ReceiveChannel<out E> { "Please note that the provided replacement does not rethrow channel's close cause as 'poll' did, " + "for the precise replacement please refer to the 'poll' documentation", replaceWith = ReplaceWith("tryReceive().getOrNull()") - ) // Warning since 1.5.0, error since 1.6.0 + ) // Warning since 1.5.0, error since 1.6.0, not hidden until 1.8+ because API is quite widespread public fun poll(): E? { val result = tryReceive() if (result.isSuccess) return result.getOrThrow() @@ -350,7 +368,7 @@ public interface ReceiveChannel<out E> { "for the detailed replacement please refer to the 'receiveOrNull' documentation", level = DeprecationLevel.ERROR, replaceWith = ReplaceWith("receiveCatching().getOrNull()") - ) // Warning since 1.3.0, error in 1.5.0, will be hidden in 1.6.0 + ) // Warning since 1.3.0, error in 1.5.0, cannot be hidden due to deprecated extensions public suspend fun receiveOrNull(): E? = receiveCatching().getOrNull() /** @@ -360,23 +378,13 @@ public interface ReceiveChannel<out E> { * * @suppress **Deprecated**: in favor of onReceiveCatching extension. */ + @Suppress("DEPRECATION_ERROR") @Deprecated( message = "Deprecated in favor of onReceiveCatching extension", level = DeprecationLevel.ERROR, replaceWith = ReplaceWith("onReceiveCatching") ) // Warning since 1.3.0, error in 1.5.0, will be hidden or removed in 1.7.0 - public val onReceiveOrNull: SelectClause1<E?> - get() { - return object : SelectClause1<E?> { - @InternalCoroutinesApi - override fun <R> registerSelectClause1(select: SelectInstance<R>, block: suspend (E?) -> R) { - onReceiveCatching.registerSelectClause1(select) { - it.exceptionOrNull()?.let { throw it } - block(it.getOrNull()) - } - } - } - } + public val onReceiveOrNull: SelectClause1<E?> get() = (this as BufferedChannel<E>).onReceiveOrNull } /** @@ -387,7 +395,7 @@ public interface ReceiveChannel<out E> { * The successful result represents a successful operation with a value of type [T], for example, * the result of [Channel.receiveCatching] operation or a successfully sent element as a result of [Channel.trySend]. * - * The failed result represents a failed operation attempt to a channel, but it doesn't necessary indicate that the channel is failed. + * The failed result represents a failed operation attempt to a channel, but it doesn't necessarily indicate that the channel is failed. * E.g. when the channel is full, [Channel.trySend] returns failed result, but the channel itself is not in the failed state. * * The closed result represents an operation attempt to a closed channel and also implies that the operation has failed. @@ -459,7 +467,6 @@ public value class ChannelResult<out T> override fun toString(): String = "Closed($cause)" } - @Suppress("NOTHING_TO_INLINE") @InternalCoroutinesApi public companion object { private val failed = Failed() @@ -523,7 +530,6 @@ public inline fun <T> ChannelResult<T>.onFailure(action: (exception: Throwable?) contract { callsInPlace(action, InvocationKind.AT_MOST_ONCE) } - @Suppress("UNCHECKED_CAST") if (holder is ChannelResult.Failed) action(exceptionOrNull()) return this } @@ -542,7 +548,6 @@ public inline fun <T> ChannelResult<T>.onClosed(action: (exception: Throwable?) contract { callsInPlace(action, InvocationKind.AT_MOST_ONCE) } - @Suppress("UNCHECKED_CAST") if (holder is ChannelResult.Closed) action(exceptionOrNull()) return this } @@ -773,26 +778,24 @@ public fun <E> Channel( when (capacity) { RENDEZVOUS -> { if (onBufferOverflow == BufferOverflow.SUSPEND) - RendezvousChannel(onUndeliveredElement) // an efficient implementation of rendezvous channel + BufferedChannel(RENDEZVOUS, onUndeliveredElement) // an efficient implementation of rendezvous channel else - ArrayChannel(1, onBufferOverflow, onUndeliveredElement) // support buffer overflow with buffered channel + ConflatedBufferedChannel(1, onBufferOverflow, onUndeliveredElement) // support buffer overflow with buffered channel } CONFLATED -> { require(onBufferOverflow == BufferOverflow.SUSPEND) { "CONFLATED capacity cannot be used with non-default onBufferOverflow" } - ConflatedChannel(onUndeliveredElement) + ConflatedBufferedChannel(1, BufferOverflow.DROP_OLDEST, onUndeliveredElement) + } + UNLIMITED -> BufferedChannel(UNLIMITED, onUndeliveredElement) // ignores onBufferOverflow: it has buffer, but it never overflows + BUFFERED -> { // uses default capacity with SUSPEND + if (onBufferOverflow == BufferOverflow.SUSPEND) BufferedChannel(CHANNEL_DEFAULT_CAPACITY, onUndeliveredElement) + else ConflatedBufferedChannel(1, onBufferOverflow, onUndeliveredElement) } - UNLIMITED -> LinkedListChannel(onUndeliveredElement) // ignores onBufferOverflow: it has buffer, but it never overflows - BUFFERED -> ArrayChannel( // uses default capacity with SUSPEND - if (onBufferOverflow == BufferOverflow.SUSPEND) CHANNEL_DEFAULT_CAPACITY else 1, - onBufferOverflow, onUndeliveredElement - ) else -> { - if (capacity == 1 && onBufferOverflow == BufferOverflow.DROP_OLDEST) - ConflatedChannel(onUndeliveredElement) // conflated implementation is more efficient but appears to work in the same way - else - ArrayChannel(capacity, onBufferOverflow, onUndeliveredElement) + if (onBufferOverflow === BufferOverflow.SUSPEND) BufferedChannel(capacity, onUndeliveredElement) + else ConflatedBufferedChannel(capacity, onBufferOverflow, onUndeliveredElement) } } diff --git a/kotlinx-coroutines-core/common/src/channels/ChannelCoroutine.kt b/kotlinx-coroutines-core/common/src/channels/ChannelCoroutine.kt index 57b2797d..3fcf388a 100644 --- a/kotlinx-coroutines-core/common/src/channels/ChannelCoroutine.kt +++ b/kotlinx-coroutines-core/common/src/channels/ChannelCoroutine.kt @@ -7,7 +7,6 @@ package kotlinx.coroutines.channels import kotlinx.coroutines.* import kotlin.coroutines.* -@Suppress("DEPRECATION") internal open class ChannelCoroutine<E>( parentContext: CoroutineContext, protected val _channel: Channel<E>, @@ -17,6 +16,7 @@ internal open class ChannelCoroutine<E>( val channel: Channel<E> get() = this + @Deprecated(level = DeprecationLevel.HIDDEN, message = "Since 1.2.0, binary compatibility with versions <= 1.1.x") override fun cancel() { cancelInternal(defaultCancellationException()) } diff --git a/kotlinx-coroutines-core/common/src/channels/Channels.common.kt b/kotlinx-coroutines-core/common/src/channels/Channels.common.kt index a78e2f18..ce454ff9 100644 --- a/kotlinx-coroutines-core/common/src/channels/Channels.common.kt +++ b/kotlinx-coroutines-core/common/src/channels/Channels.common.kt @@ -3,7 +3,6 @@ */ @file:JvmMultifileClass @file:JvmName("ChannelsKt") -@file:Suppress("DEPRECATION_ERROR") @file:OptIn(ExperimentalContracts::class) package kotlinx.coroutines.channels @@ -11,7 +10,6 @@ package kotlinx.coroutines.channels import kotlinx.coroutines.* import kotlinx.coroutines.selects.* import kotlin.contracts.* -import kotlin.coroutines.* import kotlin.jvm.* internal const val DEFAULT_CLOSE_MESSAGE = "Channel was closed" @@ -23,10 +21,14 @@ internal const val DEFAULT_CLOSE_MESSAGE = "Channel was closed" * Opens subscription to this [BroadcastChannel] and makes sure that the given [block] consumes all elements * from it by always invoking [cancel][ReceiveChannel.cancel] after the execution of the block. * - * **Note: This API will become obsolete in future updates with introduction of lazy asynchronous streams.** - * See [issue #254](https://github.com/Kotlin/kotlinx.coroutines/issues/254). + * **Note: This API is obsolete since 1.5.0 and deprecated for removal since 1.7.0** + * It is replaced with [SharedFlow][kotlinx.coroutines.flow.SharedFlow]. + * + * Safe to remove in 1.9.0 as was inline before. */ @ObsoleteCoroutinesApi +@Suppress("DEPRECATION") +@Deprecated(level = DeprecationLevel.WARNING, message = "BroadcastChannel is deprecated in the favour of SharedFlow and is no longer supported") public inline fun <E, R> BroadcastChannel<E>.consume(block: ReceiveChannel<E>.() -> R): R { val channel = openSubscription() try { @@ -50,11 +52,11 @@ public inline fun <E, R> BroadcastChannel<E>.consume(block: ReceiveChannel<E>.() @Deprecated( "Deprecated in the favour of 'receiveCatching'", ReplaceWith("receiveCatching().getOrNull()"), - DeprecationLevel.ERROR + DeprecationLevel.HIDDEN ) // Warning since 1.5.0, ERROR in 1.6.0, HIDDEN in 1.7.0 -@Suppress("EXTENSION_SHADOWED_BY_MEMBER") +@Suppress("EXTENSION_SHADOWED_BY_MEMBER", "DEPRECATION_ERROR") public suspend fun <E : Any> ReceiveChannel<E>.receiveOrNull(): E? { - @Suppress("DEPRECATION", "UNCHECKED_CAST") + @Suppress("INVISIBLE_MEMBER", "INVISIBLE_REFERENCE") return (this as ReceiveChannel<E?>).receiveOrNull() } @@ -63,10 +65,10 @@ public suspend fun <E : Any> ReceiveChannel<E>.receiveOrNull(): E? { */ @Deprecated( "Deprecated in the favour of 'onReceiveCatching'", - level = DeprecationLevel.ERROR + level = DeprecationLevel.HIDDEN ) // Warning since 1.5.0, ERROR in 1.6.0, HIDDEN in 1.7.0 +@Suppress("DEPRECATION_ERROR") public fun <E : Any> ReceiveChannel<E>.onReceiveOrNull(): SelectClause1<E?> { - @Suppress("DEPRECATION", "UNCHECKED_CAST") return (this as ReceiveChannel<E?>).onReceiveOrNull } @@ -110,7 +112,6 @@ public suspend inline fun <E> ReceiveChannel<E>.consumeEach(action: (E) -> Unit) * The operation is _terminal_. * This function [consumes][ReceiveChannel.consume] all elements of the original [ReceiveChannel]. */ -@OptIn(ExperimentalStdlibApi::class) public suspend fun <E> ReceiveChannel<E>.toList(): List<E> = buildList { consumeEach { add(it) @@ -120,10 +121,10 @@ public suspend fun <E> ReceiveChannel<E>.toList(): List<E> = buildList { /** * Subscribes to this [BroadcastChannel] and performs the specified action for each received element. * - * **Note: This API will become obsolete in future updates with introduction of lazy asynchronous streams.** - * See [issue #254](https://github.com/Kotlin/kotlinx.coroutines/issues/254). + * **Note: This API is obsolete since 1.5.0 and deprecated for removal since 1.7.0** */ -@ObsoleteCoroutinesApi +@Deprecated(level = DeprecationLevel.WARNING, message = "BroadcastChannel is deprecated in the favour of SharedFlow and is no longer supported") +@Suppress("DEPRECATION") public suspend inline fun <E> BroadcastChannel<E>.consumeEach(action: (E) -> Unit): Unit = consume { for (element in this) action(element) diff --git a/kotlinx-coroutines-core/common/src/channels/ConflatedBroadcastChannel.kt b/kotlinx-coroutines-core/common/src/channels/ConflatedBroadcastChannel.kt deleted file mode 100644 index b768d7c3..00000000 --- a/kotlinx-coroutines-core/common/src/channels/ConflatedBroadcastChannel.kt +++ /dev/null @@ -1,294 +0,0 @@ -/* - * Copyright 2016-2021 JetBrains s.r.o. Use of this source code is governed by the Apache 2.0 license. - */ - -package kotlinx.coroutines.channels - -import kotlinx.atomicfu.* -import kotlinx.coroutines.* -import kotlinx.coroutines.internal.* -import kotlinx.coroutines.intrinsics.* -import kotlinx.coroutines.selects.* -import kotlin.jvm.* - -/** - * Broadcasts the most recently sent element (aka [value]) to all [openSubscription] subscribers. - * - * Back-to-send sent elements are _conflated_ -- only the the most recently sent value is received, - * while previously sent elements **are lost**. - * Every subscriber immediately receives the most recently sent element. - * Sender to this broadcast channel never suspends and [trySend] always succeeds. - * - * A secondary constructor can be used to create an instance of this class that already holds a value. - * This channel is also created by `BroadcastChannel(Channel.CONFLATED)` factory function invocation. - * - * This implementation is fully lock-free. In this implementation - * [opening][openSubscription] and [closing][ReceiveChannel.cancel] subscription takes O(N) time, where N is the - * number of subscribers. - * - * **Note: This API is obsolete since 1.5.0.** It will be deprecated with warning in 1.6.0 - * and with error in 1.7.0. It is replaced with [StateFlow][kotlinx.coroutines.flow.StateFlow]. - */ -@ObsoleteCoroutinesApi -public class ConflatedBroadcastChannel<E>() : BroadcastChannel<E> { - /** - * Creates an instance of this class that already holds a value. - * - * It is as a shortcut to creating an instance with a default constructor and - * immediately sending an element: `ConflatedBroadcastChannel().apply { offer(value) }`. - */ - public constructor(value: E) : this() { - _state.lazySet(State<E>(value, null)) - } - - private val _state = atomic<Any>(INITIAL_STATE) // State | Closed - private val _updating = atomic(0) - // State transitions: null -> handler -> HANDLER_INVOKED - private val onCloseHandler = atomic<Any?>(null) - - private companion object { - private val CLOSED = Closed(null) - private val UNDEFINED = Symbol("UNDEFINED") - private val INITIAL_STATE = State<Any?>(UNDEFINED, null) - } - - private class State<E>( - @JvmField val value: Any?, // UNDEFINED | E - @JvmField val subscribers: Array<Subscriber<E>>? - ) - - private class Closed(@JvmField val closeCause: Throwable?) { - val sendException: Throwable get() = closeCause ?: ClosedSendChannelException(DEFAULT_CLOSE_MESSAGE) - val valueException: Throwable get() = closeCause ?: IllegalStateException(DEFAULT_CLOSE_MESSAGE) - } - - /** - * The most recently sent element to this channel. - * - * Access to this property throws [IllegalStateException] when this class is constructed without - * initial value and no value was sent yet or if it was [closed][close] without a cause. - * It throws the original [close][SendChannel.close] cause exception if the channel has _failed_. - */ - @Suppress("UNCHECKED_CAST") - public val value: E get() { - _state.loop { state -> - when (state) { - is Closed -> throw state.valueException - is State<*> -> { - if (state.value === UNDEFINED) throw IllegalStateException("No value") - return state.value as E - } - else -> error("Invalid state $state") - } - } - } - - /** - * The most recently sent element to this channel or `null` when this class is constructed without - * initial value and no value was sent yet or if it was [closed][close]. - */ - public val valueOrNull: E? get() = when (val state = _state.value) { - is Closed -> null - is State<*> -> UNDEFINED.unbox<E?>(state.value) - else -> error("Invalid state $state") - } - - public override val isClosedForSend: Boolean get() = _state.value is Closed - - @Suppress("UNCHECKED_CAST") - public override fun openSubscription(): ReceiveChannel<E> { - val subscriber = Subscriber(this) - _state.loop { state -> - when (state) { - is Closed -> { - subscriber.close(state.closeCause) - return subscriber - } - is State<*> -> { - if (state.value !== UNDEFINED) - subscriber.offerInternal(state.value as E) - val update = State(state.value, addSubscriber((state as State<E>).subscribers, subscriber)) - if (_state.compareAndSet(state, update)) - return subscriber - } - else -> error("Invalid state $state") - } - } - } - - @Suppress("UNCHECKED_CAST") - private fun closeSubscriber(subscriber: Subscriber<E>) { - _state.loop { state -> - when (state) { - is Closed -> return - is State<*> -> { - val update = State(state.value, removeSubscriber((state as State<E>).subscribers!!, subscriber)) - if (_state.compareAndSet(state, update)) - return - } - else -> error("Invalid state $state") - } - } - } - - private fun addSubscriber(list: Array<Subscriber<E>>?, subscriber: Subscriber<E>): Array<Subscriber<E>> { - if (list == null) return Array(1) { subscriber } - return list + subscriber - } - - @Suppress("UNCHECKED_CAST") - private fun removeSubscriber(list: Array<Subscriber<E>>, subscriber: Subscriber<E>): Array<Subscriber<E>>? { - val n = list.size - val i = list.indexOf(subscriber) - assert { i >= 0 } - if (n == 1) return null - val update = arrayOfNulls<Subscriber<E>>(n - 1) - list.copyInto( - destination = update, - endIndex = i - ) - list.copyInto( - destination = update, - destinationOffset = i, - startIndex = i + 1 - ) - return update as Array<Subscriber<E>> - } - - @Suppress("UNCHECKED_CAST") - public override fun close(cause: Throwable?): Boolean { - _state.loop { state -> - when (state) { - is Closed -> return false - is State<*> -> { - val update = if (cause == null) CLOSED else Closed(cause) - if (_state.compareAndSet(state, update)) { - (state as State<E>).subscribers?.forEach { it.close(cause) } - invokeOnCloseHandler(cause) - return true - } - } - else -> error("Invalid state $state") - } - } - } - - private fun invokeOnCloseHandler(cause: Throwable?) { - val handler = onCloseHandler.value - if (handler !== null && handler !== HANDLER_INVOKED - && onCloseHandler.compareAndSet(handler, HANDLER_INVOKED)) { - @Suppress("UNCHECKED_CAST") - (handler as Handler)(cause) - } - } - - override fun invokeOnClose(handler: Handler) { - // Intricate dance for concurrent invokeOnClose and close - if (!onCloseHandler.compareAndSet(null, handler)) { - val value = onCloseHandler.value - if (value === HANDLER_INVOKED) { - throw IllegalStateException("Another handler was already registered and successfully invoked") - } else { - throw IllegalStateException("Another handler was already registered: $value") - } - } else { - val state = _state.value - if (state is Closed && onCloseHandler.compareAndSet(handler, HANDLER_INVOKED)) { - (handler)(state.closeCause) - } - } - } - - /** - * @suppress This method has bad semantics when cause is not a [CancellationException]. Use [cancel]. - */ - @Deprecated(level = DeprecationLevel.HIDDEN, message = "Since 1.2.0, binary compatibility with versions <= 1.1.x") - public override fun cancel(cause: Throwable?): Boolean = close(cause) - - /** - * Cancels this conflated broadcast channel with an optional cause, same as [close]. - * This function closes the channel with - * the specified cause (unless it was already closed), - * and [cancels][ReceiveChannel.cancel] all open subscriptions. - * A cause can be used to specify an error message or to provide other details on - * a cancellation reason for debugging purposes. - */ - public override fun cancel(cause: CancellationException?) { - close(cause) - } - - /** - * Sends the value to all subscribed receives and stores this value as the most recent state for - * future subscribers. This implementation never suspends. - * It throws exception if the channel [isClosedForSend] (see [close] for details). - */ - public override suspend fun send(element: E) { - offerInternal(element)?.let { throw it.sendException } - } - - /** - * Sends the value to all subscribed receives and stores this value as the most recent state for - * future subscribers. This implementation always returns either successful result - * or closed with an exception. - */ - public override fun trySend(element: E): ChannelResult<Unit> { - offerInternal(element)?.let { return ChannelResult.closed(it.sendException) } - return ChannelResult.success(Unit) - } - - @Suppress("UNCHECKED_CAST") - private fun offerInternal(element: E): Closed? { - // If some other thread is updating the state in its offer operation we assume that our offer had linearized - // before that offer (we lost) and that offer overwrote us and conflated our offer. - if (!_updating.compareAndSet(0, 1)) return null - try { - _state.loop { state -> - when (state) { - is Closed -> return state - is State<*> -> { - val update = State(element, (state as State<E>).subscribers) - if (_state.compareAndSet(state, update)) { - // Note: Using offerInternal here to ignore the case when this subscriber was - // already concurrently closed (assume the close had conflated our offer for this - // particular subscriber). - state.subscribers?.forEach { it.offerInternal(element) } - return null - } - } - else -> error("Invalid state $state") - } - } - } finally { - _updating.value = 0 // reset the updating flag to zero even when something goes wrong - } - } - - public override val onSend: SelectClause2<E, SendChannel<E>> - get() = object : SelectClause2<E, SendChannel<E>> { - override fun <R> registerSelectClause2(select: SelectInstance<R>, param: E, block: suspend (SendChannel<E>) -> R) { - registerSelectSend(select, param, block) - } - } - - private fun <R> registerSelectSend(select: SelectInstance<R>, element: E, block: suspend (SendChannel<E>) -> R) { - if (!select.trySelect()) return - offerInternal(element)?.let { - select.resumeSelectWithException(it.sendException) - return - } - block.startCoroutineUnintercepted(receiver = this, completion = select.completion) - } - - private class Subscriber<E>( - private val broadcastChannel: ConflatedBroadcastChannel<E> - ) : ConflatedChannel<E>(null), ReceiveChannel<E> { - - override fun onCancelIdempotent(wasClosed: Boolean) { - if (wasClosed) { - broadcastChannel.closeSubscriber(this) - } - } - - public override fun offerInternal(element: E): Any = super.offerInternal(element) - } -} diff --git a/kotlinx-coroutines-core/common/src/channels/ConflatedBufferedChannel.kt b/kotlinx-coroutines-core/common/src/channels/ConflatedBufferedChannel.kt new file mode 100644 index 00000000..69903072 --- /dev/null +++ b/kotlinx-coroutines-core/common/src/channels/ConflatedBufferedChannel.kt @@ -0,0 +1,118 @@ +/* + * Copyright 2016-2021 JetBrains s.r.o. Use of this source code is governed by the Apache 2.0 license. + */ + +package kotlinx.coroutines.channels + +import kotlinx.atomicfu.* +import kotlinx.coroutines.channels.BufferOverflow.* +import kotlinx.coroutines.channels.ChannelResult.Companion.closed +import kotlinx.coroutines.channels.ChannelResult.Companion.success +import kotlinx.coroutines.internal.* +import kotlinx.coroutines.internal.OnUndeliveredElement +import kotlinx.coroutines.selects.* +import kotlin.coroutines.* + +/** + * This is a special [BufferedChannel] extension that supports [DROP_OLDEST] and [DROP_LATEST] + * strategies for buffer overflowing. This implementation ensures that `send(e)` never suspends, + * either extracting the first element ([DROP_OLDEST]) or dropping the sending one ([DROP_LATEST]) + * when the channel capacity exceeds. + */ +internal open class ConflatedBufferedChannel<E>( + private val capacity: Int, + private val onBufferOverflow: BufferOverflow, + onUndeliveredElement: OnUndeliveredElement<E>? = null +) : BufferedChannel<E>(capacity = capacity, onUndeliveredElement = onUndeliveredElement) { + init { + require(onBufferOverflow !== SUSPEND) { + "This implementation does not support suspension for senders, use ${BufferedChannel::class.simpleName} instead" + } + require(capacity >= 1) { + "Buffered channel capacity must be at least 1, but $capacity was specified" + } + } + + override val isConflatedDropOldest: Boolean + get() = onBufferOverflow == DROP_OLDEST + + override suspend fun send(element: E) { + // Should never suspend, implement via `trySend(..)`. + trySendImpl(element, isSendOp = true).onClosed { // fails only when this channel is closed. + onUndeliveredElement?.callUndeliveredElementCatchingException(element)?.let { + it.addSuppressed(sendException) + throw it + } + throw sendException + } + } + + override suspend fun sendBroadcast(element: E): Boolean { + // Should never suspend, implement via `trySend(..)`. + trySendImpl(element, isSendOp = true) // fails only when this channel is closed. + .onSuccess { return true } + return false + } + + override fun trySend(element: E): ChannelResult<Unit> = trySendImpl(element, isSendOp = false) + + private fun trySendImpl(element: E, isSendOp: Boolean) = + if (onBufferOverflow === DROP_LATEST) trySendDropLatest(element, isSendOp) + else trySendDropOldest(element) + + private fun trySendDropLatest(element: E, isSendOp: Boolean): ChannelResult<Unit> { + // Try to send the element without suspension. + val result = super.trySend(element) + // Complete on success or if this channel is closed. + if (result.isSuccess || result.isClosed) return result + // This channel is full. Drop the sending element. + // Call the `onUndeliveredElement` lambda ONLY for 'send()' invocations, + // for 'trySend()' it is responsibility of the caller + if (isSendOp) { + onUndeliveredElement?.callUndeliveredElementCatchingException(element)?.let { + throw it + } + } + return success(Unit) + } + + private fun trySendDropOldest(element: E): ChannelResult<Unit> = + sendImpl( // <-- this is an inline function + element = element, + // Put the element into the logical buffer even + // if this channel is already full, the `onSuspend` + // callback below extract the first (oldest) element. + waiter = BUFFERED, + // Finish successfully when a rendezvous has happened + // or the element has been buffered. + onRendezvousOrBuffered = { return success(Unit) }, + // In case the algorithm decided to suspend, the element + // was added to the buffer. However, as the buffer is now + // overflowed, the first (oldest) element has to be extracted. + onSuspend = { segm, i -> + dropFirstElementUntilTheSpecifiedCellIsInTheBuffer(segm.id * SEGMENT_SIZE + i) + return success(Unit) + }, + // If the channel is closed, return the corresponding result. + onClosed = { return closed(sendException) } + ) + + @Suppress("UNCHECKED_CAST") + override fun registerSelectForSend(select: SelectInstance<*>, element: Any?) { + // The plain `send(..)` operation never suspends. Thus, either this + // attempt to send the element succeeds or the channel is closed. + // In any case, complete this `select` in the registration phase. + trySend(element as E).let { + it.onSuccess { + select.selectInRegistrationPhase(Unit) + return + }.onClosed { + select.selectInRegistrationPhase(CHANNEL_CLOSED) + return + } + } + error("unreachable") + } + + override fun shouldSendSuspend() = false // never suspends. +} diff --git a/kotlinx-coroutines-core/common/src/channels/ConflatedChannel.kt b/kotlinx-coroutines-core/common/src/channels/ConflatedChannel.kt deleted file mode 100644 index 177e80cb..00000000 --- a/kotlinx-coroutines-core/common/src/channels/ConflatedChannel.kt +++ /dev/null @@ -1,143 +0,0 @@ -/* - * Copyright 2016-2021 JetBrains s.r.o. Use of this source code is governed by the Apache 2.0 license. - */ - -package kotlinx.coroutines.channels - -import kotlinx.coroutines.* -import kotlinx.coroutines.internal.* -import kotlinx.coroutines.selects.* - -/** - * Channel that buffers at most one element and conflates all subsequent `send` and `trySend` invocations, - * so that the receiver always gets the most recently sent element. - * Back-to-send sent elements are _conflated_ -- only the most recently sent element is received, - * while previously sent elements **are lost**. - * Sender to this channel never suspends and [trySend] always succeeds. - * - * This channel is created by `Channel(Channel.CONFLATED)` factory function invocation. - */ -internal open class ConflatedChannel<E>(onUndeliveredElement: OnUndeliveredElement<E>?) : AbstractChannel<E>(onUndeliveredElement) { - protected final override val isBufferAlwaysEmpty: Boolean get() = false - protected final override val isBufferEmpty: Boolean get() = lock.withLock { value === EMPTY } - protected final override val isBufferAlwaysFull: Boolean get() = false - protected final override val isBufferFull: Boolean get() = false - - override val isEmpty: Boolean get() = lock.withLock { isEmptyImpl } - - private val lock = ReentrantLock() - - private var value: Any? = EMPTY - - // result is `OFFER_SUCCESS | Closed` - protected override fun offerInternal(element: E): Any { - var receive: ReceiveOrClosed<E>? = null - lock.withLock { - closedForSend?.let { return it } - // if there is no element written in buffer - if (value === EMPTY) { - // check for receivers that were waiting on the empty buffer - loop@ while(true) { - receive = takeFirstReceiveOrPeekClosed() ?: break@loop // break when no receivers queued - if (receive is Closed) { - return receive!! - } - val token = receive!!.tryResumeReceive(element, null) - if (token != null) { - assert { token === RESUME_TOKEN } - return@withLock - } - } - } - updateValueLocked(element)?.let { throw it } - return OFFER_SUCCESS - } - // breaks here if offer meets receiver - receive!!.completeResumeReceive(element) - return receive!!.offerResult - } - - // result is `ALREADY_SELECTED | OFFER_SUCCESS | Closed` - protected override fun offerSelectInternal(element: E, select: SelectInstance<*>): Any { - var receive: ReceiveOrClosed<E>? = null - lock.withLock { - closedForSend?.let { return it } - if (value === EMPTY) { - loop@ while(true) { - val offerOp = describeTryOffer(element) - val failure = select.performAtomicTrySelect(offerOp) - when { - failure == null -> { // offered successfully - receive = offerOp.result - return@withLock - } - failure === OFFER_FAILED -> break@loop // cannot offer -> Ok to queue to buffer - failure === RETRY_ATOMIC -> {} // retry - failure === ALREADY_SELECTED || failure is Closed<*> -> return failure - else -> error("performAtomicTrySelect(describeTryOffer) returned $failure") - } - } - } - // try to select sending this element to buffer - if (!select.trySelect()) { - return ALREADY_SELECTED - } - updateValueLocked(element)?.let { throw it } - return OFFER_SUCCESS - } - // breaks here if offer meets receiver - receive!!.completeResumeReceive(element) - return receive!!.offerResult - } - - // result is `E | POLL_FAILED | Closed` - protected override fun pollInternal(): Any? { - var result: Any? = null - lock.withLock { - if (value === EMPTY) return closedForSend ?: POLL_FAILED - result = value - value = EMPTY - } - return result - } - - // result is `E | POLL_FAILED | Closed` - protected override fun pollSelectInternal(select: SelectInstance<*>): Any? { - var result: Any? = null - lock.withLock { - if (value === EMPTY) return closedForSend ?: POLL_FAILED - if (!select.trySelect()) - return ALREADY_SELECTED - result = value - value = EMPTY - } - return result - } - - protected override fun onCancelIdempotent(wasClosed: Boolean) { - var undeliveredElementException: UndeliveredElementException? = null // resource cancel exception - lock.withLock { - undeliveredElementException = updateValueLocked(EMPTY) - } - super.onCancelIdempotent(wasClosed) - undeliveredElementException?.let { throw it } // throw UndeliveredElementException at the end if there was one - } - - @Suppress("UNCHECKED_CAST") - private fun updateValueLocked(element: Any?): UndeliveredElementException? { - val old = value - val undeliveredElementException = if (old === EMPTY) null else - onUndeliveredElement?.callUndeliveredElementCatchingException(old as E) - value = element - return undeliveredElementException - } - - override fun enqueueReceiveInternal(receive: Receive<E>): Boolean = lock.withLock { - super.enqueueReceiveInternal(receive) - } - - // ------ debug ------ - - override val bufferDebugString: String - get() = lock.withLock { "(value=$value)" } -} diff --git a/kotlinx-coroutines-core/common/src/channels/LinkedListChannel.kt b/kotlinx-coroutines-core/common/src/channels/LinkedListChannel.kt deleted file mode 100644 index b5f607b2..00000000 --- a/kotlinx-coroutines-core/common/src/channels/LinkedListChannel.kt +++ /dev/null @@ -1,76 +0,0 @@ -/* - * Copyright 2016-2021 JetBrains s.r.o. Use of this source code is governed by the Apache 2.0 license. - */ - -package kotlinx.coroutines.channels - -import kotlinx.coroutines.internal.* -import kotlinx.coroutines.selects.* - -/** - * Channel with linked-list buffer of a unlimited capacity (limited only by available memory). - * Sender to this channel never suspends and [trySend] always succeeds. - * - * This channel is created by `Channel(Channel.UNLIMITED)` factory function invocation. - * - * This implementation is fully lock-free. - * - * @suppress **This an internal API and should not be used from general code.** - */ -internal open class LinkedListChannel<E>(onUndeliveredElement: OnUndeliveredElement<E>?) : AbstractChannel<E>(onUndeliveredElement) { - protected final override val isBufferAlwaysEmpty: Boolean get() = true - protected final override val isBufferEmpty: Boolean get() = true - protected final override val isBufferAlwaysFull: Boolean get() = false - protected final override val isBufferFull: Boolean get() = false - - // result is always `OFFER_SUCCESS | Closed` - protected override fun offerInternal(element: E): Any { - while (true) { - val result = super.offerInternal(element) - when { - result === OFFER_SUCCESS -> return OFFER_SUCCESS - result === OFFER_FAILED -> { // try to buffer - when (val sendResult = sendBuffered(element)) { - null -> return OFFER_SUCCESS - is Closed<*> -> return sendResult - } - // otherwise there was receiver in queue, retry super.offerInternal - } - result is Closed<*> -> return result - else -> error("Invalid offerInternal result $result") - } - } - } - - // result is always `ALREADY_SELECTED | OFFER_SUCCESS | Closed`. - protected override fun offerSelectInternal(element: E, select: SelectInstance<*>): Any { - while (true) { - val result = if (hasReceiveOrClosed) - super.offerSelectInternal(element, select) else - (select.performAtomicTrySelect(describeSendBuffered(element)) ?: OFFER_SUCCESS) - when { - result === ALREADY_SELECTED -> return ALREADY_SELECTED - result === OFFER_SUCCESS -> return OFFER_SUCCESS - result === OFFER_FAILED -> {} // retry - result === RETRY_ATOMIC -> {} // retry - result is Closed<*> -> return result - else -> error("Invalid result $result") - } - } - } - - override fun onCancelIdempotentList(list: InlineList<Send>, closed: Closed<*>) { - var undeliveredElementException: UndeliveredElementException? = null - list.forEachReversed { - when (it) { - is SendBuffered<*> -> { - @Suppress("UNCHECKED_CAST") - undeliveredElementException = onUndeliveredElement?.callUndeliveredElementCatchingException(it.element as E, undeliveredElementException) - } - else -> it.resumeSendClosed(closed) - } - } - undeliveredElementException?.let { throw it } // throw UndeliveredElementException at the end if there was one - } -} - diff --git a/kotlinx-coroutines-core/common/src/channels/RendezvousChannel.kt b/kotlinx-coroutines-core/common/src/channels/RendezvousChannel.kt deleted file mode 100644 index e8ade513..00000000 --- a/kotlinx-coroutines-core/common/src/channels/RendezvousChannel.kt +++ /dev/null @@ -1,23 +0,0 @@ -/* - * Copyright 2016-2021 JetBrains s.r.o. Use of this source code is governed by the Apache 2.0 license. - */ - -package kotlinx.coroutines.channels - -import kotlinx.coroutines.internal.* - -/** - * Rendezvous channel. This channel does not have any buffer at all. An element is transferred from sender - * to receiver only when [send] and [receive] invocations meet in time (rendezvous), so [send] suspends - * until another coroutine invokes [receive] and [receive] suspends until another coroutine invokes [send]. - * - * Use `Channel()` factory function to conveniently create an instance of rendezvous channel. - * - * This implementation is fully lock-free. - **/ -internal open class RendezvousChannel<E>(onUndeliveredElement: OnUndeliveredElement<E>?) : AbstractChannel<E>(onUndeliveredElement) { - protected final override val isBufferAlwaysEmpty: Boolean get() = true - protected final override val isBufferEmpty: Boolean get() = true - protected final override val isBufferAlwaysFull: Boolean get() = true - protected final override val isBufferFull: Boolean get() = true -} diff --git a/kotlinx-coroutines-core/common/src/flow/Builders.kt b/kotlinx-coroutines-core/common/src/flow/Builders.kt index c4b55e10..350cc4be 100644 --- a/kotlinx-coroutines-core/common/src/flow/Builders.kt +++ b/kotlinx-coroutines-core/common/src/flow/Builders.kt @@ -65,7 +65,6 @@ private class SafeFlow<T>(private val block: suspend FlowCollector<T>.() -> Unit /** * Creates a _cold_ flow that produces a single value from the given functional type. */ -@FlowPreview public fun <T> (() -> T).asFlow(): Flow<T> = flow { emit(invoke()) } @@ -80,7 +79,6 @@ public fun <T> (() -> T).asFlow(): Flow<T> = flow { * fun remoteCallFlow(): Flow<R> = ::remoteCall.asFlow() * ``` */ -@FlowPreview public fun <T> (suspend () -> T).asFlow(): Flow<T> = flow { emit(invoke()) } diff --git a/kotlinx-coroutines-core/common/src/flow/Channels.kt b/kotlinx-coroutines-core/common/src/flow/Channels.kt index 51ed4270..a9566628 100644 --- a/kotlinx-coroutines-core/common/src/flow/Channels.kt +++ b/kotlinx-coroutines-core/common/src/flow/Channels.kt @@ -31,35 +31,10 @@ public suspend fun <T> FlowCollector<T>.emitAll(channel: ReceiveChannel<T>): Uni private suspend fun <T> FlowCollector<T>.emitAllImpl(channel: ReceiveChannel<T>, consume: Boolean) { ensureActive() - // Manually inlined "consumeEach" implementation that does not use iterator but works via "receiveCatching". - // It has smaller and more efficient spilled state which also allows to implement a manual kludge to - // fix retention of the last emitted value. - // See https://youtrack.jetbrains.com/issue/KT-16222 - // See https://github.com/Kotlin/kotlinx.coroutines/issues/1333 var cause: Throwable? = null try { - while (true) { - // :KLUDGE: This "run" call is resolved to an extension function "run" and forces the size of - // spilled state to increase by an additional slot, so there are 4 object local variables spilled here - // which makes the size of spill state equal to the 4 slots that are spilled around subsequent "emit" - // call, ensuring that the previously emitted value is not retained in the state while receiving - // the next one. - // L$0 <- this - // L$1 <- channel - // L$2 <- cause - // L$3 <- this$run (actually equal to this) - val result = run { channel.receiveCatching() } - if (result.isClosed) { - result.exceptionOrNull()?.let { throw it } - break // returns normally when result.closeCause == null - } - // result is spilled here to the coroutine state and retained after the call, even though - // it is not actually needed in the next loop iteration. - // L$0 <- this - // L$1 <- channel - // L$2 <- cause - // L$3 <- result - emit(result.getOrThrow()) + for (element in channel) { + emit(element) } } catch (e: Throwable) { cause = e @@ -169,11 +144,12 @@ private class ChannelAsFlow<T>( * 2) Flow consumer completes normally when the original channel completes (~is closed) normally. * 3) If the flow consumer fails with an exception, subscription is cancelled. */ +@Suppress("DEPRECATION") @Deprecated( - level = DeprecationLevel.WARNING, + level = DeprecationLevel.ERROR, message = "'BroadcastChannel' is obsolete and all corresponding operators are deprecated " + "in the favour of StateFlow and SharedFlow" -) // Since 1.5.0, was @FlowPreview, safe to remove in 1.7.0 +) // Since 1.5.0, ERROR since 1.7.0, was @FlowPreview, safe to remove in 1.8.0 public fun <T> BroadcastChannel<T>.asFlow(): Flow<T> = flow { emitAll(openSubscription()) } @@ -193,7 +169,6 @@ public fun <T> BroadcastChannel<T>.asFlow(): Flow<T> = flow { * default and to control what happens when data is produced faster than it is consumed, * that is to control backpressure behavior. */ -@FlowPreview public fun <T> Flow<T>.produceIn( scope: CoroutineScope ): ReceiveChannel<T> = diff --git a/kotlinx-coroutines-core/common/src/flow/Flow.kt b/kotlinx-coroutines-core/common/src/flow/Flow.kt index 3520c48b..92006d46 100644 --- a/kotlinx-coroutines-core/common/src/flow/Flow.kt +++ b/kotlinx-coroutines-core/common/src/flow/Flow.kt @@ -221,7 +221,7 @@ public interface Flow<out T> { * } * ``` */ -@FlowPreview +@ExperimentalCoroutinesApi public abstract class AbstractFlow<T> : Flow<T>, CancellableFlow<T> { public final override suspend fun collect(collector: FlowCollector<T>) { diff --git a/kotlinx-coroutines-core/common/src/flow/SharedFlow.kt b/kotlinx-coroutines-core/common/src/flow/SharedFlow.kt index 0a291f25..b4833fea 100644 --- a/kotlinx-coroutines-core/common/src/flow/SharedFlow.kt +++ b/kotlinx-coroutines-core/common/src/flow/SharedFlow.kt @@ -10,7 +10,6 @@ import kotlinx.coroutines.flow.internal.* import kotlinx.coroutines.internal.* import kotlin.coroutines.* import kotlin.jvm.* -import kotlin.native.concurrent.* /** * A _hot_ [Flow] that shares emitted values among all its collectors in a broadcast fashion, so that all collectors @@ -710,7 +709,6 @@ internal open class SharedFlowImpl<T>( } } -@SharedImmutable @JvmField internal val NO_VALUE = Symbol("NO_VALUE") diff --git a/kotlinx-coroutines-core/common/src/flow/StateFlow.kt b/kotlinx-coroutines-core/common/src/flow/StateFlow.kt index be6cbd6b..2f3de156 100644 --- a/kotlinx-coroutines-core/common/src/flow/StateFlow.kt +++ b/kotlinx-coroutines-core/common/src/flow/StateFlow.kt @@ -10,7 +10,6 @@ import kotlinx.coroutines.channels.* import kotlinx.coroutines.flow.internal.* import kotlinx.coroutines.internal.* import kotlin.coroutines.* -import kotlin.native.concurrent.* /** * A [SharedFlow] that represents a read-only state with a single updatable data [value] that emits updates @@ -146,8 +145,9 @@ public interface StateFlow<out T> : SharedFlow<T> { * A mutable [StateFlow] that provides a setter for [value]. * An instance of `MutableStateFlow` with the given initial `value` can be created using * `MutableStateFlow(value)` constructor function. - * + * See the [StateFlow] documentation for details on state flows. + * Note that all emission-related operators, such as [value]'s setter, [emit], and [tryEmit], are conflated using [Any.equals]. * * ### Not stable for inheritance * @@ -238,10 +238,8 @@ public inline fun <T> MutableStateFlow<T>.update(function: (T) -> T) { // ------------------------------------ Implementation ------------------------------------ -@SharedImmutable private val NONE = Symbol("NONE") -@SharedImmutable private val PENDING = Symbol("PENDING") // StateFlow slots are allocated for its collectors @@ -321,8 +319,8 @@ private class StateFlowImpl<T>( updateState(expect ?: NULL, update ?: NULL) private fun updateState(expectedState: Any?, newState: Any): Boolean { - var curSequence = 0 - var curSlots: Array<StateFlowSlot?>? = this.slots // benign race, we will not use it + var curSequence: Int + var curSlots: Array<StateFlowSlot?>? // benign race, we will not use it synchronized(this) { val oldState = _state.value if (expectedState != null && oldState != expectedState) return false // CAS support diff --git a/kotlinx-coroutines-core/common/src/flow/internal/AbstractSharedFlow.kt b/kotlinx-coroutines-core/common/src/flow/internal/AbstractSharedFlow.kt index 39ca9839..5dcf0d01 100644 --- a/kotlinx-coroutines-core/common/src/flow/internal/AbstractSharedFlow.kt +++ b/kotlinx-coroutines-core/common/src/flow/internal/AbstractSharedFlow.kt @@ -9,10 +9,8 @@ import kotlinx.coroutines.flow.* import kotlinx.coroutines.internal.* import kotlin.coroutines.* import kotlin.jvm.* -import kotlin.native.concurrent.* @JvmField -@SharedImmutable internal val EMPTY_RESUMES = arrayOfNulls<Continuation<Unit>?>(0) internal abstract class AbstractSharedFlowSlot<F> { @@ -21,7 +19,6 @@ internal abstract class AbstractSharedFlowSlot<F> { } internal abstract class AbstractSharedFlow<S : AbstractSharedFlowSlot<*>> : SynchronizedObject() { - @Suppress("UNCHECKED_CAST") protected var slots: Array<S?>? = null // allocated when needed private set protected var nCollectors = 0 // number of allocated (!free) slots @@ -44,7 +41,7 @@ internal abstract class AbstractSharedFlow<S : AbstractSharedFlowSlot<*>> : Sync @Suppress("UNCHECKED_CAST") protected fun allocateSlot(): S { // Actually create slot under lock - var subscriptionCount: SubscriptionCountStateFlow? = null + val subscriptionCount: SubscriptionCountStateFlow? val slot = synchronized(this) { val slots = when (val curSlots = slots) { null -> createSlotArray(2).also { slots = it } @@ -75,7 +72,7 @@ internal abstract class AbstractSharedFlow<S : AbstractSharedFlowSlot<*>> : Sync @Suppress("UNCHECKED_CAST") protected fun freeSlot(slot: S) { // Release slot under lock - var subscriptionCount: SubscriptionCountStateFlow? = null + val subscriptionCount: SubscriptionCountStateFlow? val resumes = synchronized(this) { nCollectors-- subscriptionCount = _subscriptionCount // retrieve under lock if initialized diff --git a/kotlinx-coroutines-core/common/src/flow/internal/ChannelFlow.kt b/kotlinx-coroutines-core/common/src/flow/internal/ChannelFlow.kt index 0efe5f86..8610cf5a 100644 --- a/kotlinx-coroutines-core/common/src/flow/internal/ChannelFlow.kt +++ b/kotlinx-coroutines-core/common/src/flow/internal/ChannelFlow.kt @@ -161,7 +161,7 @@ internal abstract class ChannelFlowOperator<S, T>( // Fast-path: When channel creation is optional (flowOn/flowWith operators without buffer) if (capacity == Channel.OPTIONAL_CHANNEL) { val collectContext = coroutineContext - val newContext = collectContext + context // compute resulting collect context + val newContext = collectContext.newCoroutineContext(context) // compute resulting collect context // #1: If the resulting context happens to be the same as it was -- fallback to plain collect if (newContext == collectContext) return flowCollect(collector) diff --git a/kotlinx-coroutines-core/common/src/flow/internal/Combine.kt b/kotlinx-coroutines-core/common/src/flow/internal/Combine.kt index c924c090..63ea55a5 100644 --- a/kotlinx-coroutines-core/common/src/flow/internal/Combine.kt +++ b/kotlinx-coroutines-core/common/src/flow/internal/Combine.kt @@ -1,7 +1,7 @@ /* * Copyright 2016-2021 JetBrains s.r.o. Use of this source code is governed by the Apache 2.0 license. */ -@file:Suppress("UNCHECKED_CAST", "NON_APPLICABLE_CALL_FOR_BUILDER_INFERENCE") // KT-32203 +@file:Suppress("UNCHECKED_CAST") // KT-32203 package kotlinx.coroutines.flow.internal @@ -9,9 +9,6 @@ import kotlinx.coroutines.* import kotlinx.coroutines.channels.* import kotlinx.coroutines.flow.* import kotlinx.coroutines.internal.* -import kotlin.coroutines.* -import kotlin.coroutines.intrinsics.* - private typealias Update = IndexedValue<Any?> @PublishedApi diff --git a/kotlinx-coroutines-core/common/src/flow/internal/NullSurrogate.kt b/kotlinx-coroutines-core/common/src/flow/internal/NullSurrogate.kt index c7327bd3..7b59e271 100644 --- a/kotlinx-coroutines-core/common/src/flow/internal/NullSurrogate.kt +++ b/kotlinx-coroutines-core/common/src/flow/internal/NullSurrogate.kt @@ -6,7 +6,6 @@ package kotlinx.coroutines.flow.internal import kotlinx.coroutines.internal.* import kotlin.jvm.* -import kotlin.native.concurrent.* /** * This value is used a a surrogate `null` value when needed. @@ -14,7 +13,6 @@ import kotlin.native.concurrent.* * Its usage typically are paired with [Symbol.unbox] usages. */ @JvmField -@SharedImmutable internal val NULL = Symbol("NULL") /** @@ -22,7 +20,6 @@ internal val NULL = Symbol("NULL") * It should never leak to the outside world. */ @JvmField -@SharedImmutable internal val UNINITIALIZED = Symbol("UNINITIALIZED") /* @@ -30,5 +27,4 @@ internal val UNINITIALIZED = Symbol("UNINITIALIZED") * It should never leak to the outside world. */ @JvmField -@SharedImmutable internal val DONE = Symbol("DONE") diff --git a/kotlinx-coroutines-core/common/src/flow/operators/Delay.kt b/kotlinx-coroutines-core/common/src/flow/operators/Delay.kt index 258dc3ee..37505dc1 100644 --- a/kotlinx-coroutines-core/common/src/flow/operators/Delay.kt +++ b/kotlinx-coroutines-core/common/src/flow/operators/Delay.kt @@ -18,7 +18,6 @@ import kotlin.time.* <!--- TEST_NAME FlowDelayTest --> <!--- PREFIX .*-duration-.* ----- INCLUDE .*-duration-.* -import kotlin.time.* ----- INCLUDE .* import kotlinx.coroutines.* import kotlinx.coroutines.flow.* @@ -345,3 +344,61 @@ internal fun CoroutineScope.fixedPeriodTicker(delayMillis: Long, initialDelayMil */ @FlowPreview public fun <T> Flow<T>.sample(period: Duration): Flow<T> = sample(period.toDelayMillis()) + +/** + * Returns a flow that will emit a [TimeoutCancellationException] if the upstream doesn't emit an item within the given time. + * + * Example: + * + * ```kotlin + * flow { + * emit(1) + * delay(100) + * emit(2) + * delay(100) + * emit(3) + * delay(1000) + * emit(4) + * }.timeout(100.milliseconds).catch { + * emit(-1) // Item to emit on timeout + * }.onEach { + * delay(300) // This will not cause a timeout + * } + * ``` + * <!--- KNIT example-timeout-duration-01.kt --> + * + * produces the following emissions + * + * ```text + * 1, 2, 3, -1 + * ``` + * <!--- TEST --> + * + * Note that delaying on the downstream doesn't trigger the timeout. + * + * @param timeout Timeout duration. If non-positive, the flow is timed out immediately + */ +@FlowPreview +public fun <T> Flow<T>.timeout( + timeout: Duration +): Flow<T> = timeoutInternal(timeout) + +private fun <T> Flow<T>.timeoutInternal( + timeout: Duration +): Flow<T> = scopedFlow { downStream -> + if (timeout <= Duration.ZERO) throw TimeoutCancellationException("Timed out immediately") + val values = buffer(Channel.RENDEZVOUS).produceIn(this) + whileSelect { + values.onReceiveCatching { value -> + value.onSuccess { + downStream.emit(it) + }.onClosed { + return@onReceiveCatching false + } + return@onReceiveCatching true + } + onTimeout(timeout) { + throw TimeoutCancellationException("Timed out waiting for $timeout") + } + } +} diff --git a/kotlinx-coroutines-core/common/src/flow/operators/Distinct.kt b/kotlinx-coroutines-core/common/src/flow/operators/Distinct.kt index f211a1b2..006d37e1 100644 --- a/kotlinx-coroutines-core/common/src/flow/operators/Distinct.kt +++ b/kotlinx-coroutines-core/common/src/flow/operators/Distinct.kt @@ -7,10 +7,8 @@ package kotlinx.coroutines.flow -import kotlinx.coroutines.* import kotlinx.coroutines.flow.internal.* import kotlin.jvm.* -import kotlin.native.concurrent.* /** * Returns flow where all subsequent repetitions of the same value are filtered out. @@ -45,10 +43,8 @@ public fun <T> Flow<T>.distinctUntilChanged(areEquivalent: (old: T, new: T) -> B public fun <T, K> Flow<T>.distinctUntilChangedBy(keySelector: (T) -> K): Flow<T> = distinctUntilChangedBy(keySelector = keySelector, areEquivalent = defaultAreEquivalent) -@SharedImmutable private val defaultKeySelector: (Any?) -> Any? = { it } -@SharedImmutable private val defaultAreEquivalent: (Any?, Any?) -> Boolean = { old, new -> old == new } /** diff --git a/kotlinx-coroutines-core/common/src/flow/operators/Lint.kt b/kotlinx-coroutines-core/common/src/flow/operators/Lint.kt index b140e628..d7fec3fe 100644 --- a/kotlinx-coroutines-core/common/src/flow/operators/Lint.kt +++ b/kotlinx-coroutines-core/common/src/flow/operators/Lint.kt @@ -148,6 +148,16 @@ public suspend inline fun <T> SharedFlow<T>.toList(): List<T> = (this as Flow<T>).toList() /** + * A specialized version of [Flow.toList] that returns [Nothing] + * to indicate that [SharedFlow] collection never completes. + */ +@InlineOnly +public suspend inline fun <T> SharedFlow<T>.toList(destination: MutableList<T>): Nothing { + (this as Flow<T>).toList(destination) + throw IllegalStateException("this code is supposed to be unreachable") +} + +/** * @suppress */ @Suppress("DeprecatedCallableAddReplaceWith") @@ -160,6 +170,16 @@ public suspend inline fun <T> SharedFlow<T>.toSet(): Set<T> = (this as Flow<T>).toSet() /** + * A specialized version of [Flow.toSet] that returns [Nothing] + * to indicate that [SharedFlow] collection never completes. + */ +@InlineOnly +public suspend inline fun <T> SharedFlow<T>.toSet(destination: MutableSet<T>): Nothing { + (this as Flow<T>).toSet(destination) + throw IllegalStateException("this code is supposed to be unreachable") +} + +/** * @suppress */ @Suppress("DeprecatedCallableAddReplaceWith") diff --git a/kotlinx-coroutines-core/common/src/flow/operators/Merge.kt b/kotlinx-coroutines-core/common/src/flow/operators/Merge.kt index 35c44d08..dfd08b8f 100644 --- a/kotlinx-coroutines-core/common/src/flow/operators/Merge.kt +++ b/kotlinx-coroutines-core/common/src/flow/operators/Merge.kt @@ -17,6 +17,7 @@ import kotlinx.coroutines.flow.internal.unsafeFlow as flow /** * Name of the property that defines the value of [DEFAULT_CONCURRENCY]. + * This is a preview API and can be changed in a backwards-incompatible manner within a single release. */ @FlowPreview public const val DEFAULT_CONCURRENCY_PROPERTY_NAME: String = "kotlinx.coroutines.flow.defaultConcurrency" @@ -24,9 +25,11 @@ public const val DEFAULT_CONCURRENCY_PROPERTY_NAME: String = "kotlinx.coroutines /** * Default concurrency limit that is used by [flattenMerge] and [flatMapMerge] operators. * It is 16 by default and can be changed on JVM using [DEFAULT_CONCURRENCY_PROPERTY_NAME] property. + * This is a preview API and can be changed in a backwards-incompatible manner within a single release. */ @FlowPreview -public val DEFAULT_CONCURRENCY: Int = systemProp(DEFAULT_CONCURRENCY_PROPERTY_NAME, +public val DEFAULT_CONCURRENCY: Int = systemProp( + DEFAULT_CONCURRENCY_PROPERTY_NAME, 16, 1, Int.MAX_VALUE ) @@ -39,7 +42,7 @@ public val DEFAULT_CONCURRENCY: Int = systemProp(DEFAULT_CONCURRENCY_PROPERTY_NA * Note that even though this operator looks very familiar, we discourage its usage in a regular application-specific flows. * Most likely, suspending operation in [map] operator will be sufficient and linear transformations are much easier to reason about. */ -@FlowPreview +@ExperimentalCoroutinesApi public fun <T, R> Flow<T>.flatMapConcat(transform: suspend (value: T) -> Flow<R>): Flow<R> = map(transform).flattenConcat() @@ -63,7 +66,7 @@ public fun <T, R> Flow<T>.flatMapConcat(transform: suspend (value: T) -> Flow<R> * @param concurrency controls the number of in-flight flows, at most [concurrency] flows are collected * at the same time. By default, it is equal to [DEFAULT_CONCURRENCY]. */ -@FlowPreview +@ExperimentalCoroutinesApi public fun <T, R> Flow<T>.flatMapMerge( concurrency: Int = DEFAULT_CONCURRENCY, transform: suspend (value: T) -> Flow<R> @@ -75,7 +78,7 @@ public fun <T, R> Flow<T>.flatMapMerge( * * Inner flows are collected by this operator *sequentially*. */ -@FlowPreview +@ExperimentalCoroutinesApi public fun <T> Flow<Flow<T>>.flattenConcat(): Flow<T> = flow { collect { value -> emitAll(value) } } @@ -132,7 +135,7 @@ public fun <T> merge(vararg flows: Flow<T>): Flow<T> = flows.asIterable().merge( * @param concurrency controls the number of in-flight flows, at most [concurrency] flows are collected * at the same time. By default, it is equal to [DEFAULT_CONCURRENCY]. */ -@FlowPreview +@ExperimentalCoroutinesApi public fun <T> Flow<Flow<T>>.flattenMerge(concurrency: Int = DEFAULT_CONCURRENCY): Flow<T> { require(concurrency > 0) { "Expected positive concurrency level, but had $concurrency" } return if (concurrency == 1) flattenConcat() else ChannelFlowMerge(this, concurrency) diff --git a/kotlinx-coroutines-core/common/src/flow/operators/Transform.kt b/kotlinx-coroutines-core/common/src/flow/operators/Transform.kt index 0f9e3959..4123aed0 100644 --- a/kotlinx-coroutines-core/common/src/flow/operators/Transform.kt +++ b/kotlinx-coroutines-core/common/src/flow/operators/Transform.kt @@ -11,6 +11,7 @@ package kotlinx.coroutines.flow import kotlinx.coroutines.* import kotlinx.coroutines.flow.internal.* import kotlin.jvm.* +import kotlin.reflect.* import kotlinx.coroutines.flow.internal.unsafeFlow as flow import kotlinx.coroutines.flow.unsafeTransform as transform @@ -35,6 +36,11 @@ public inline fun <T> Flow<T>.filterNot(crossinline predicate: suspend (T) -> Bo public inline fun <reified R> Flow<*>.filterIsInstance(): Flow<R> = filter { it is R } as Flow<R> /** + * Returns a flow containing only values that are instances of the given [klass]. + */ +public fun <R : Any> Flow<*>.filterIsInstance(klass: KClass<R>): Flow<R> = filter { klass.isInstance(it) } as Flow<R> + +/** * Returns a flow containing only values of the original flow that are not null. */ public fun <T: Any> Flow<T?>.filterNotNull(): Flow<T> = transform<T?, T> { value -> @@ -81,7 +87,7 @@ public fun <T> Flow<T>.onEach(action: suspend (T) -> Unit): Flow<T> = transform * ``` * flowOf(1, 2, 3).scan(emptyList<Int>()) { acc, value -> acc + value }.toList() * ``` - * will produce `[], [1], [1, 2], [1, 2, 3]`. + * will produce `[[], [1], [1, 2], [1, 2, 3]]`. * * This function is an alias to [runningFold] operator. */ @@ -94,7 +100,7 @@ public fun <T, R> Flow<T>.scan(initial: R, @BuilderInference operation: suspend * ``` * flowOf(1, 2, 3).runningFold(emptyList<Int>()) { acc, value -> acc + value }.toList() * ``` - * will produce `[], [1], [1, 2], [1, 2, 3]`. + * will produce `[[], [1], [1, 2], [1, 2, 3]]`. */ public fun <T, R> Flow<T>.runningFold(initial: R, @BuilderInference operation: suspend (accumulator: R, value: T) -> R): Flow<R> = flow { var accumulator: R = initial diff --git a/kotlinx-coroutines-core/common/src/flow/terminal/Count.kt b/kotlinx-coroutines-core/common/src/flow/terminal/Count.kt index 5eb99fc8..a15567e2 100644 --- a/kotlinx-coroutines-core/common/src/flow/terminal/Count.kt +++ b/kotlinx-coroutines-core/common/src/flow/terminal/Count.kt @@ -7,7 +7,6 @@ package kotlinx.coroutines.flow -import kotlinx.coroutines.* import kotlin.jvm.* /** diff --git a/kotlinx-coroutines-core/common/src/flow/terminal/Reduce.kt b/kotlinx-coroutines-core/common/src/flow/terminal/Reduce.kt index 1794c9f4..557c095a 100644 --- a/kotlinx-coroutines-core/common/src/flow/terminal/Reduce.kt +++ b/kotlinx-coroutines-core/common/src/flow/terminal/Reduce.kt @@ -49,7 +49,7 @@ public suspend inline fun <T, R> Flow<T>.fold( /** * The terminal operator that awaits for one and only one value to be emitted. - * Throws [NoSuchElementException] for empty flow and [IllegalStateException] for flow + * Throws [NoSuchElementException] for empty flow and [IllegalArgumentException] for flow * that contains more than one element. */ public suspend fun <T> Flow<T>.single(): T { diff --git a/kotlinx-coroutines-core/common/src/internal/ArrayQueue.kt b/kotlinx-coroutines-core/common/src/internal/ArrayQueue.kt deleted file mode 100644 index 6b994b68..00000000 --- a/kotlinx-coroutines-core/common/src/internal/ArrayQueue.kt +++ /dev/null @@ -1,52 +0,0 @@ -/* - * Copyright 2016-2021 JetBrains s.r.o. Use of this source code is governed by the Apache 2.0 license. - */ - -package kotlinx.coroutines.internal - -internal open class ArrayQueue<T : Any> { - private var elements = arrayOfNulls<Any>(16) - private var head = 0 - private var tail = 0 - - val isEmpty: Boolean get() = head == tail - - public fun addLast(element: T) { - elements[tail] = element - tail = (tail + 1) and elements.size - 1 - if (tail == head) ensureCapacity() - } - - @Suppress("UNCHECKED_CAST") - public fun removeFirstOrNull(): T? { - if (head == tail) return null - val element = elements[head] - elements[head] = null - head = (head + 1) and elements.size - 1 - return element as T - } - - public fun clear() { - head = 0 - tail = 0 - elements = arrayOfNulls(elements.size) - } - - private fun ensureCapacity() { - val currentSize = elements.size - val newCapacity = currentSize shl 1 - val newElements = arrayOfNulls<Any>(newCapacity) - elements.copyInto( - destination = newElements, - startIndex = head - ) - elements.copyInto( - destination = newElements, - destinationOffset = elements.size - head, - endIndex = head - ) - elements = newElements - head = 0 - tail = currentSize - } -} diff --git a/kotlinx-coroutines-core/common/src/internal/Atomic.kt b/kotlinx-coroutines-core/common/src/internal/Atomic.kt index cf43764c..ff4320e0 100644 --- a/kotlinx-coroutines-core/common/src/internal/Atomic.kt +++ b/kotlinx-coroutines-core/common/src/internal/Atomic.kt @@ -8,7 +8,6 @@ package kotlinx.coroutines.internal import kotlinx.atomicfu.atomic import kotlinx.coroutines.* import kotlin.jvm.* -import kotlin.native.concurrent.* /** * The most abstract operation that can be in process. Other threads observing an instance of this @@ -30,15 +29,8 @@ public abstract class OpDescriptor { abstract val atomicOp: AtomicOp<*>? override fun toString(): String = "$classSimpleName@$hexAddress" // debug - - fun isEarlierThan(that: OpDescriptor): Boolean { - val thisOp = atomicOp ?: return false - val thatOp = that.atomicOp ?: return false - return thisOp.opSequence < thatOp.opSequence - } } -@SharedImmutable @JvmField internal val NO_DECISION: Any = Symbol("NO_DECISION") @@ -57,25 +49,9 @@ internal val NO_DECISION: Any = Symbol("NO_DECISION") public abstract class AtomicOp<in T> : OpDescriptor() { private val _consensus = atomic<Any?>(NO_DECISION) - // Returns NO_DECISION when there is not decision yet - val consensus: Any? get() = _consensus.value - - val isDecided: Boolean get() = _consensus.value !== NO_DECISION - - /** - * Sequence number of this multi-word operation for deadlock resolution. - * An operation with lower number aborts itself with (using [RETRY_ATOMIC] error symbol) if it encounters - * the need to help the operation with higher sequence number and then restarts - * (using higher `opSequence` to ensure progress). - * Simple operations that cannot get into the deadlock always return zero here. - * - * See https://github.com/Kotlin/kotlinx.coroutines/issues/504 - */ - open val opSequence: Long get() = 0L - override val atomicOp: AtomicOp<*> get() = this - fun decide(decision: Any?): Any? { + private fun decide(decision: Any?): Any? { assert { decision !== NO_DECISION } val current = _consensus.value if (current !== NO_DECISION) return current @@ -100,22 +76,3 @@ public abstract class AtomicOp<in T> : OpDescriptor() { return decision } } - -/** - * A part of multi-step atomic operation [AtomicOp]. - * - * @suppress **This is unstable API and it is subject to change.** - */ -public abstract class AtomicDesc { - lateinit var atomicOp: AtomicOp<*> // the reference to parent atomicOp, init when AtomicOp is created - abstract fun prepare(op: AtomicOp<*>): Any? // returns `null` if prepared successfully - abstract fun complete(op: AtomicOp<*>, failure: Any?) // decision == null if success -} - -/** - * It is returned as an error by [AtomicOp] implementations when they detect potential deadlock - * using [AtomicOp.opSequence] numbers. - */ -@JvmField -@SharedImmutable -internal val RETRY_ATOMIC: Any = Symbol("RETRY_ATOMIC") diff --git a/kotlinx-coroutines-core/common/src/internal/Concurrent.common.kt b/kotlinx-coroutines-core/common/src/internal/Concurrent.common.kt index fb254a0e..848a42c8 100644 --- a/kotlinx-coroutines-core/common/src/internal/Concurrent.common.kt +++ b/kotlinx-coroutines-core/common/src/internal/Concurrent.common.kt @@ -4,21 +4,9 @@ package kotlinx.coroutines.internal -/** - * Special kind of list intended to be used as collection of subscribers in `ArrayBroadcastChannel` - * On JVM it's CopyOnWriteList and on JS it's MutableList. - * - * Note that this alias is intentionally not named as CopyOnWriteList to avoid accidental misusage outside of the ArrayBroadcastChannel - */ -internal typealias SubscribersList<E> = MutableList<E> - -@Deprecated(message = "Implementation of this primitive is tailored to specific ArrayBroadcastChannel usages on K/N " + - "and K/JS platforms and it is unsafe to use it anywhere else") -internal expect fun <E> subscriberList(): SubscribersList<E> - internal expect class ReentrantLock() { fun tryLock(): Boolean - fun unlock(): Unit + fun unlock() } internal expect inline fun <T> ReentrantLock.withLock(action: () -> T): T diff --git a/kotlinx-coroutines-core/common/src/internal/ConcurrentLinkedList.kt b/kotlinx-coroutines-core/common/src/internal/ConcurrentLinkedList.kt index 638ec432..ecfafcf4 100644 --- a/kotlinx-coroutines-core/common/src/internal/ConcurrentLinkedList.kt +++ b/kotlinx-coroutines-core/common/src/internal/ConcurrentLinkedList.kt @@ -6,16 +6,16 @@ package kotlinx.coroutines.internal import kotlinx.atomicfu.* import kotlinx.coroutines.* +import kotlin.coroutines.* import kotlin.jvm.* -import kotlin.native.concurrent.SharedImmutable /** * Returns the first segment `s` with `s.id >= id` or `CLOSED` * if all the segments in this linked list have lower `id`, and the list is closed for further segment additions. */ -private inline fun <S : Segment<S>> S.findSegmentInternal( +internal fun <S : Segment<S>> S.findSegmentInternal( id: Long, - createNewSegment: (id: Long, prev: S?) -> S + createNewSegment: (id: Long, prev: S) -> S ): SegmentOrClosed<S> { /* Go through `next` references and add new segments if needed, similarly to the `push` in the Michael-Scott @@ -23,7 +23,7 @@ private inline fun <S : Segment<S>> S.findSegmentInternal( added, so the algorithm just uses it. This way, only one segment with each id can be added. */ var cur: S = this - while (cur.id < id || cur.removed) { + while (cur.id < id || cur.isRemoved) { val next = cur.nextOrIfClosed { return SegmentOrClosed(CLOSED) } if (next != null) { // there is a next node -- move there cur = next @@ -31,7 +31,7 @@ private inline fun <S : Segment<S>> S.findSegmentInternal( } val newTail = createNewSegment(cur.id + 1, cur) if (cur.trySetNext(newTail)) { // successfully added new node -- move there - if (cur.removed) cur.remove() + if (cur.isRemoved) cur.remove() cur = newTail } } @@ -41,8 +41,8 @@ private inline fun <S : Segment<S>> S.findSegmentInternal( /** * Returns `false` if the segment `to` is logically removed, `true` on a successful update. */ -@Suppress("NOTHING_TO_INLINE") // Must be inline because it is an AtomicRef extension -private inline fun <S : Segment<S>> AtomicRef<S>.moveForward(to: S): Boolean = loop { cur -> +@Suppress("NOTHING_TO_INLINE", "RedundantNullableReturnType") // Must be inline because it is an AtomicRef extension +internal inline fun <S : Segment<S>> AtomicRef<S>.moveForward(to: S): Boolean = loop { cur -> if (cur.id >= to.id) return true if (!to.tryIncPointers()) return false if (compareAndSet(cur, to)) { // the segment is moved @@ -63,10 +63,11 @@ private inline fun <S : Segment<S>> AtomicRef<S>.moveForward(to: S): Boolean = l * Returns the segment `s` with `s.id >= id` or `CLOSED` if all the segments in this linked list have lower `id`, * and the list is closed. */ +@Suppress("NOTHING_TO_INLINE") internal inline fun <S : Segment<S>> AtomicRef<S>.findSegmentAndMoveForward( id: Long, startFrom: S, - createNewSegment: (id: Long, prev: S?) -> S + noinline createNewSegment: (id: Long, prev: S) -> S ): SegmentOrClosed<S> { while (true) { val s = startFrom.findSegmentInternal(id, createNewSegment) @@ -137,47 +138,49 @@ internal abstract class ConcurrentLinkedListNode<N : ConcurrentLinkedListNode<N> /** * This property indicates whether the current node is logically removed. - * The expected use-case is removing the node logically (so that [removed] becomes true), + * The expected use-case is removing the node logically (so that [isRemoved] becomes true), * and invoking [remove] after that. Note that this implementation relies on the contract * that the physical tail cannot be logically removed. Please, do not break this contract; * otherwise, memory leaks and unexpected behavior can occur. */ - abstract val removed: Boolean + abstract val isRemoved: Boolean /** * Removes this node physically from this linked list. The node should be - * logically removed (so [removed] returns `true`) at the point of invocation. + * logically removed (so [isRemoved] returns `true`) at the point of invocation. */ fun remove() { - assert { removed } // The node should be logically removed at first. - assert { !isTail } // The physical tail cannot be removed. + assert { isRemoved || isTail } // The node should be logically removed at first. + // The physical tail cannot be removed. Instead, we remove it when + // a new segment is added and this segment is not the tail one anymore. + if (isTail) return while (true) { // Read `next` and `prev` pointers ignoring logically removed nodes. - val prev = leftmostAliveNode - val next = rightmostAliveNode + val prev = aliveSegmentLeft + val next = aliveSegmentRight // Link `next` and `prev`. - next._prev.value = prev + next._prev.update { if (it === null) null else prev } if (prev !== null) prev._next.value = next // Checks that prev and next are still alive. - if (next.removed) continue - if (prev !== null && prev.removed) continue + if (next.isRemoved && !next.isTail) continue + if (prev !== null && prev.isRemoved) continue // This node is removed. return } } - private val leftmostAliveNode: N? get() { + private val aliveSegmentLeft: N? get() { var cur = prev - while (cur !== null && cur.removed) + while (cur !== null && cur.isRemoved) cur = cur._prev.value return cur } - private val rightmostAliveNode: N get() { + private val aliveSegmentRight: N get() { assert { !isTail } // Should not be invoked on the tail node var cur = next!! - while (cur.removed) - cur = cur.next!! + while (cur.isRemoved) + cur = cur.next ?: return cur return cur } } @@ -186,13 +189,26 @@ internal abstract class ConcurrentLinkedListNode<N : ConcurrentLinkedListNode<N> * Each segment in the list has a unique id and is created by the provided to [findSegmentAndMoveForward] method. * Essentially, this is a node in the Michael-Scott queue algorithm, * but with maintaining [prev] pointer for efficient [remove] implementation. + * + * NB: this class cannot be public or leak into user's code as public type as [CancellableContinuationImpl] + * instance-check it and uses a separate code-path for that. */ -internal abstract class Segment<S : Segment<S>>(val id: Long, prev: S?, pointers: Int): ConcurrentLinkedListNode<S>(prev) { +internal abstract class Segment<S : Segment<S>>( + @JvmField val id: Long, prev: S?, pointers: Int +) : ConcurrentLinkedListNode<S>(prev), + // Segments typically store waiting continuations. Thus, on cancellation, the corresponding + // slot should be cleaned and the segment should be removed if it becomes full of cancelled cells. + // To install such a handler efficiently, without creating an extra object, we allow storing + // segments as cancellation handlers in [CancellableContinuationImpl] state, putting the slot + // index in another field. The details are here: https://github.com/Kotlin/kotlinx.coroutines/pull/3084. + // For that, we need segments to implement this internal marker interface. + NotCompleted +{ /** - * This property should return the maximal number of slots in this segment, + * This property should return the number of slots in this segment, * it is used to define whether the segment is logically removed. */ - abstract val maxSlots: Int + abstract val numberOfSlots: Int /** * Numbers of cleaned slots (the lowest bits) and AtomicRef pointers to this segment (the highest bits) @@ -200,23 +216,35 @@ internal abstract class Segment<S : Segment<S>>(val id: Long, prev: S?, pointers private val cleanedAndPointers = atomic(pointers shl POINTERS_SHIFT) /** - * The segment is considered as removed if all the slots are cleaned. - * There are no pointers to this segment from outside, and - * it is not a physical tail in the linked list of segments. + * The segment is considered as removed if all the slots are cleaned + * and there are no pointers to this segment from outside. */ - override val removed get() = cleanedAndPointers.value == maxSlots && !isTail + override val isRemoved get() = cleanedAndPointers.value == numberOfSlots && !isTail // increments the number of pointers if this segment is not logically removed. - internal fun tryIncPointers() = cleanedAndPointers.addConditionally(1 shl POINTERS_SHIFT) { it != maxSlots || isTail } + internal fun tryIncPointers() = cleanedAndPointers.addConditionally(1 shl POINTERS_SHIFT) { it != numberOfSlots || isTail } // returns `true` if this segment is logically removed after the decrement. - internal fun decPointers() = cleanedAndPointers.addAndGet(-(1 shl POINTERS_SHIFT)) == maxSlots && !isTail + internal fun decPointers() = cleanedAndPointers.addAndGet(-(1 shl POINTERS_SHIFT)) == numberOfSlots && !isTail + + /** + * This function is invoked on continuation cancellation when this segment + * with the specified [index] are installed as cancellation handler via + * `SegmentDisposable.disposeOnCancellation(Segment, Int)`. + * + * @param index the index under which the sement registered itself in the continuation. + * Indicies are opaque and arithmetics or numeric intepretation is not allowed on them, + * as they may encode additional metadata. + * @param cause the cause of the cancellation, with the same semantics as [CancellableContinuation.invokeOnCancellation] + * @param context the context of the cancellable continuation the segment was registered in + */ + abstract fun onCancellation(index: Int, cause: Throwable?, context: CoroutineContext) /** * Invoked on each slot clean-up; should not be invoked twice for the same slot. */ fun onSlotCleaned() { - if (cleanedAndPointers.incrementAndGet() == maxSlots && !isTail) remove() + if (cleanedAndPointers.incrementAndGet() == numberOfSlots) remove() } } @@ -237,5 +265,4 @@ internal value class SegmentOrClosed<S : Segment<S>>(private val value: Any?) { private const val POINTERS_SHIFT = 16 -@SharedImmutable private val CLOSED = Symbol("CLOSED") diff --git a/kotlinx-coroutines-core/common/src/internal/CoroutineExceptionHandlerImpl.common.kt b/kotlinx-coroutines-core/common/src/internal/CoroutineExceptionHandlerImpl.common.kt new file mode 100644 index 00000000..3f5925a3 --- /dev/null +++ b/kotlinx-coroutines-core/common/src/internal/CoroutineExceptionHandlerImpl.common.kt @@ -0,0 +1,72 @@ +/* + * Copyright 2016-2022 JetBrains s.r.o. Use of this source code is governed by the Apache 2.0 license. + */ + +package kotlinx.coroutines.internal + +import kotlinx.coroutines.* +import kotlin.coroutines.* + +/** + * The list of globally installed [CoroutineExceptionHandler] instances that will be notified of any exceptions that + * were not processed in any other manner. + */ +internal expect val platformExceptionHandlers: Collection<CoroutineExceptionHandler> + +/** + * Ensures that the given [callback] is present in the [platformExceptionHandlers] list. + */ +internal expect fun ensurePlatformExceptionHandlerLoaded(callback: CoroutineExceptionHandler) + +/** + * The platform-dependent global exception handler, used so that the exception is logged at least *somewhere*. + */ +internal expect fun propagateExceptionFinalResort(exception: Throwable) + +/** + * Deal with exceptions that happened in coroutines and weren't programmatically dealt with. + * + * First, it notifies every [CoroutineExceptionHandler] in the [platformExceptionHandlers] list. + * If one of them throws [ExceptionSuccessfullyProcessed], it means that that handler believes that the exception was + * dealt with sufficiently well and doesn't need any further processing. + * Otherwise, the platform-dependent global exception handler is also invoked. + */ +internal fun handleUncaughtCoroutineException(context: CoroutineContext, exception: Throwable) { + // use additional extension handlers + for (handler in platformExceptionHandlers) { + try { + handler.handleException(context, exception) + } catch (_: ExceptionSuccessfullyProcessed) { + return + } catch (t: Throwable) { + propagateExceptionFinalResort(handlerException(exception, t)) + } + } + + try { + exception.addSuppressed(DiagnosticCoroutineContextException(context)) + } catch (e: Throwable) { + // addSuppressed is never user-defined and cannot normally throw with the only exception being OOM + // we do ignore that just in case to definitely deliver the exception + } + propagateExceptionFinalResort(exception) +} + +/** + * Private exception that is added to suppressed exceptions of the original exception + * when it is reported to the last-ditch current thread 'uncaughtExceptionHandler'. + * + * The purpose of this exception is to add an otherwise inaccessible diagnostic information and to + * be able to poke the context of the failing coroutine in the debugger. + */ +internal expect class DiagnosticCoroutineContextException(context: CoroutineContext) : RuntimeException + +/** + * A dummy exception that signifies that the exception was successfully processed by the handler and no further + * action is required. + * + * Would be nicer if [CoroutineExceptionHandler] could return a boolean, but that would be a breaking change. + * For now, we will take solace in knowledge that such exceptions are exceedingly rare, even rarer than globally + * uncaught exceptions in general. + */ +internal object ExceptionSuccessfullyProcessed : Exception() diff --git a/kotlinx-coroutines-core/common/src/internal/DispatchedContinuation.kt b/kotlinx-coroutines-core/common/src/internal/DispatchedContinuation.kt index c689a381..0241c8df 100644 --- a/kotlinx-coroutines-core/common/src/internal/DispatchedContinuation.kt +++ b/kotlinx-coroutines-core/common/src/internal/DispatchedContinuation.kt @@ -8,16 +8,15 @@ import kotlinx.atomicfu.* import kotlinx.coroutines.* import kotlin.coroutines.* import kotlin.jvm.* -import kotlin.native.concurrent.* -@SharedImmutable private val UNDEFINED = Symbol("UNDEFINED") -@SharedImmutable @JvmField internal val REUSABLE_CLAIMED = Symbol("REUSABLE_CLAIMED") +@PublishedApi internal class DispatchedContinuation<in T>( - @JvmField val dispatcher: CoroutineDispatcher, + @JvmField internal val dispatcher: CoroutineDispatcher, + // Used by the IDEA debugger via reflection and must be kept binary-compatible, see KTIJ-24102 @JvmField val continuation: Continuation<T> ) : DispatchedTask<T>(MODE_UNINITIALIZED), CoroutineStackFrame, Continuation<T> by continuation { @JvmField @@ -61,7 +60,7 @@ internal class DispatchedContinuation<in T>( private val reusableCancellableContinuation: CancellableContinuationImpl<*>? get() = _reusableCancellableContinuation.value as? CancellableContinuationImpl<*> - fun isReusable(): Boolean { + internal fun isReusable(): Boolean { /* Invariant: caller.resumeMode.isReusableMode * Reusability control: @@ -75,13 +74,13 @@ internal class DispatchedContinuation<in T>( * Awaits until previous call to `suspendCancellableCoroutineReusable` will * stop mutating cached instance */ - fun awaitReusability() { + internal fun awaitReusability() { _reusableCancellableContinuation.loop { if (it !== REUSABLE_CLAIMED) return } } - fun release() { + internal fun release() { /* * Called from `releaseInterceptedContinuation`, can be concurrent with * the code in `getResult` right after `trySuspend` returned `true`, so we have @@ -96,7 +95,7 @@ internal class DispatchedContinuation<in T>( * so all cancellations will be postponed. */ @Suppress("UNCHECKED_CAST") - fun claimReusableCancellableContinuation(): CancellableContinuationImpl<T>? { + internal fun claimReusableCancellableContinuation(): CancellableContinuationImpl<T>? { /* * Transitions: * 1) `null` -> claimed, caller will instantiate CC instance @@ -145,7 +144,7 @@ internal class DispatchedContinuation<in T>( * * See [CancellableContinuationImpl.getResult]. */ - fun tryReleaseClaimedContinuation(continuation: CancellableContinuation<*>): Throwable? { + internal fun tryReleaseClaimedContinuation(continuation: CancellableContinuation<*>): Throwable? { _reusableCancellableContinuation.loop { state -> // not when(state) to avoid Intrinsics.equals call when { @@ -165,7 +164,7 @@ internal class DispatchedContinuation<in T>( * Tries to postpone cancellation if reusable CC is currently in [REUSABLE_CLAIMED] state. * Returns `true` if cancellation is (or previously was) postponed, `false` otherwise. */ - fun postponeCancellation(cause: Throwable): Boolean { + internal fun postponeCancellation(cause: Throwable): Boolean { _reusableCancellableContinuation.loop { state -> when (state) { REUSABLE_CLAIMED -> { @@ -211,7 +210,7 @@ internal class DispatchedContinuation<in T>( // We inline it to save an entry on the stack in cases where it shows (unconfined dispatcher) // It is used only in Continuation<T>.resumeCancellableWith @Suppress("NOTHING_TO_INLINE") - inline fun resumeCancellableWith( + internal inline fun resumeCancellableWith( result: Result<T>, noinline onCancellation: ((cause: Throwable) -> Unit)? ) { @@ -238,8 +237,9 @@ internal class DispatchedContinuation<in T>( } } + // inline here is to save us an entry on the stack for the sake of better stacktraces @Suppress("NOTHING_TO_INLINE") - inline fun resumeCancelled(state: Any?): Boolean { + internal inline fun resumeCancelled(state: Any?): Boolean { val job = context[Job] if (job != null && !job.isActive) { val cause = job.getCancellationException() @@ -250,8 +250,8 @@ internal class DispatchedContinuation<in T>( return false } - @Suppress("NOTHING_TO_INLINE") // we need it inline to save us an entry on the stack - inline fun resumeUndispatchedWith(result: Result<T>) { + @Suppress("NOTHING_TO_INLINE") + internal inline fun resumeUndispatchedWith(result: Result<T>) { withContinuationContext(continuation, countOrElement) { continuation.resumeWith(result) } diff --git a/kotlinx-coroutines-core/common/src/internal/DispatchedTask.kt b/kotlinx-coroutines-core/common/src/internal/DispatchedTask.kt index d982f95b..fc22e024 100644 --- a/kotlinx-coroutines-core/common/src/internal/DispatchedTask.kt +++ b/kotlinx-coroutines-core/common/src/internal/DispatchedTask.kt @@ -47,7 +47,9 @@ internal const val MODE_UNINITIALIZED = -1 internal val Int.isCancellableMode get() = this == MODE_CANCELLABLE || this == MODE_CANCELLABLE_REUSABLE internal val Int.isReusableMode get() = this == MODE_CANCELLABLE_REUSABLE -internal abstract class DispatchedTask<in T>( +@PublishedApi +internal abstract class DispatchedTask<in T> internal constructor( + // Used by the IDEA debugger via reflection and must be kept binary-compatible, see KTIJ-24102 @JvmField public var resumeMode: Int ) : SchedulerTask() { internal abstract val delegate: Continuation<T> @@ -78,7 +80,7 @@ internal abstract class DispatchedTask<in T>( internal open fun getExceptionalResult(state: Any?): Throwable? = (state as? CompletedExceptionally)?.cause - public final override fun run() { + final override fun run() { assert { resumeMode != MODE_UNINITIALIZED } // should have been set before dispatching val taskContext = this.taskContext var fatalException: Throwable? = null @@ -134,7 +136,7 @@ internal abstract class DispatchedTask<in T>( * Fatal exception handling can be intercepted with [CoroutineExceptionHandler] element in the context of * a failed coroutine, but such exceptions should be reported anyway. */ - public fun handleFatalException(exception: Throwable?, finallyException: Throwable?) { + internal fun handleFatalException(exception: Throwable?, finallyException: Throwable?) { if (exception === null && finallyException === null) return if (exception !== null && finallyException !== null) { exception.addSuppressedThrowable(finallyException) @@ -167,7 +169,6 @@ internal fun <T> DispatchedTask<T>.dispatch(mode: Int) { } } -@Suppress("UNCHECKED_CAST") internal fun <T> DispatchedTask<T>.resume(delegate: Continuation<T>, undispatched: Boolean) { // This resume is never cancellable. The result is always delivered to delegate continuation. val state = takeState() diff --git a/kotlinx-coroutines-core/common/src/internal/LimitedDispatcher.kt b/kotlinx-coroutines-core/common/src/internal/LimitedDispatcher.kt index 28f37ecf..374e53b7 100644 --- a/kotlinx-coroutines-core/common/src/internal/LimitedDispatcher.kt +++ b/kotlinx-coroutines-core/common/src/internal/LimitedDispatcher.kt @@ -4,22 +4,33 @@ package kotlinx.coroutines.internal +import kotlinx.atomicfu.* import kotlinx.coroutines.* import kotlin.coroutines.* -import kotlin.jvm.* /** * The result of .limitedParallelism(x) call, a dispatcher * that wraps the given dispatcher, but limits the parallelism level, while * trying to emulate fairness. + * + * ### Implementation details + * + * By design, 'LimitedDispatcher' never [dispatches][CoroutineDispatcher.dispatch] originally sent tasks + * to the underlying dispatcher. Instead, it maintains its own queue of tasks sent to this dispatcher and + * dispatches at most [parallelism] "worker-loop" tasks that poll the underlying queue and cooperatively preempt + * in order to avoid starvation of the underlying dispatcher. + * + * Such behavior is crucial to be compatible with any underlying dispatcher implementation without + * direct cooperation. */ internal class LimitedDispatcher( private val dispatcher: CoroutineDispatcher, private val parallelism: Int -) : CoroutineDispatcher(), Runnable, Delay by (dispatcher as? Delay ?: DefaultDelay) { +) : CoroutineDispatcher(), Delay by (dispatcher as? Delay ?: DefaultDelay) { - @Volatile - private var runningWorkers = 0 + // Atomic is necessary here for the sake of K/N memory ordering, + // there is no need in atomic operations for this property + private val runningWorkers = atomic(0) private val queue = LockFreeTaskQueue<Runnable>(singleConsumer = false) @@ -33,72 +44,88 @@ internal class LimitedDispatcher( return super.limitedParallelism(parallelism) } - override fun run() { - var fairnessCounter = 0 - while (true) { - val task = queue.removeFirstOrNull() - if (task != null) { - try { - task.run() - } catch (e: Throwable) { - handleCoroutineException(EmptyCoroutineContext, e) - } - // 16 is our out-of-thin-air constant to emulate fairness. Used in JS dispatchers as well - if (++fairnessCounter >= 16 && dispatcher.isDispatchNeeded(this)) { - // Do "yield" to let other views to execute their runnable as well - // Note that we do not decrement 'runningWorkers' as we still committed to do our part of work - dispatcher.dispatch(this, this) - return - } - continue - } - - synchronized(workerAllocationLock) { - --runningWorkers - if (queue.size == 0) return - ++runningWorkers - fairnessCounter = 0 - } - } - } - override fun dispatch(context: CoroutineContext, block: Runnable) { - dispatchInternal(block) { - dispatcher.dispatch(this, this) + dispatchInternal(block) { worker -> + dispatcher.dispatch(this, worker) } } @InternalCoroutinesApi override fun dispatchYield(context: CoroutineContext, block: Runnable) { - dispatchInternal(block) { - dispatcher.dispatchYield(this, this) + dispatchInternal(block) { worker -> + dispatcher.dispatchYield(this, worker) } } - private inline fun dispatchInternal(block: Runnable, dispatch: () -> Unit) { + /** + * Tries to dispatch the given [block]. + * If there are not enough workers, it starts a new one via [startWorker]. + */ + private inline fun dispatchInternal(block: Runnable, startWorker: (Worker) -> Unit) { // Add task to queue so running workers will be able to see that - if (addAndTryDispatching(block)) return - /* - * Protect against the race when the number of workers is enough, - * but one (because of synchronized serialization) attempts to complete, - * and we just observed the number of running workers smaller than the actual - * number (hit right between `--runningWorkers` and `++runningWorkers` in `run()`) - */ + queue.addLast(block) + if (runningWorkers.value >= parallelism) return + // allocation may fail if some workers were launched in parallel or a worker temporarily decreased + // `runningWorkers` when they observed an empty queue. if (!tryAllocateWorker()) return - dispatch() + val task = obtainTaskOrDeallocateWorker() ?: return + startWorker(Worker(task)) } + /** + * Tries to obtain the permit to start a new worker. + */ private fun tryAllocateWorker(): Boolean { synchronized(workerAllocationLock) { - if (runningWorkers >= parallelism) return false - ++runningWorkers + if (runningWorkers.value >= parallelism) return false + runningWorkers.incrementAndGet() return true } } - private fun addAndTryDispatching(block: Runnable): Boolean { - queue.addLast(block) - return runningWorkers >= parallelism + /** + * Obtains the next task from the queue, or logically deallocates the worker if the queue is empty. + */ + private fun obtainTaskOrDeallocateWorker(): Runnable? { + while (true) { + when (val nextTask = queue.removeFirstOrNull()) { + null -> synchronized(workerAllocationLock) { + runningWorkers.decrementAndGet() + if (queue.size == 0) return null + runningWorkers.incrementAndGet() + } + else -> return nextTask + } + } + } + + /** + * A worker that polls the queue and runs tasks until there are no more of them. + * + * It always stores the next task to run. This is done in order to prevent the possibility of the fairness + * re-dispatch happening when there are no more tasks in the queue. This is important because, after all the + * actual tasks are done, nothing prevents the user from closing the dispatcher and making it incorrect to + * perform any more dispatches. + */ + private inner class Worker(private var currentTask: Runnable) : Runnable { + override fun run() { + var fairnessCounter = 0 + while (true) { + try { + currentTask.run() + } catch (e: Throwable) { + handleCoroutineException(EmptyCoroutineContext, e) + } + currentTask = obtainTaskOrDeallocateWorker() ?: return + // 16 is our out-of-thin-air constant to emulate fairness. Used in JS dispatchers as well + if (++fairnessCounter >= 16 && dispatcher.isDispatchNeeded(this@LimitedDispatcher)) { + // Do "yield" to let other views execute their runnable as well + // Note that we do not decrement 'runningWorkers' as we are still committed to our part of work + dispatcher.dispatch(this@LimitedDispatcher, this) + return + } + } + } } } diff --git a/kotlinx-coroutines-core/common/src/internal/LockFreeLinkedList.common.kt b/kotlinx-coroutines-core/common/src/internal/LockFreeLinkedList.common.kt index 8b20ade1..121cdedc 100644 --- a/kotlinx-coroutines-core/common/src/internal/LockFreeLinkedList.common.kt +++ b/kotlinx-coroutines-core/common/src/internal/LockFreeLinkedList.common.kt @@ -5,8 +5,8 @@ package kotlinx.coroutines.internal +import kotlinx.coroutines.* import kotlin.jvm.* -import kotlin.native.concurrent.* /** @suppress **This is unstable API and it is subject to change.** */ public expect open class LockFreeLinkedListNode() { @@ -16,27 +16,8 @@ public expect open class LockFreeLinkedListNode() { public fun addLast(node: LockFreeLinkedListNode) public fun addOneIfEmpty(node: LockFreeLinkedListNode): Boolean public inline fun addLastIf(node: LockFreeLinkedListNode, crossinline condition: () -> Boolean): Boolean - public inline fun addLastIfPrev( - node: LockFreeLinkedListNode, - predicate: (LockFreeLinkedListNode) -> Boolean - ): Boolean - - public inline fun addLastIfPrevAndIf( - node: LockFreeLinkedListNode, - predicate: (LockFreeLinkedListNode) -> Boolean, // prev node predicate - crossinline condition: () -> Boolean // atomically checked condition - ): Boolean - public open fun remove(): Boolean - /** - * Helps fully finish [remove] operation, must be invoked after [remove] if needed. - * Ensures that traversing the list via prev pointers sees this node as removed. - * No-op on JS - */ - public fun helpRemove() - public fun removeFirstOrNull(): LockFreeLinkedListNode? - public inline fun <reified T> removeFirstIfIsInstanceOfOrPeekIf(predicate: (T) -> Boolean): T? } /** @suppress **This is unstable API and it is subject to change.** */ @@ -45,46 +26,3 @@ public expect open class LockFreeLinkedListHead() : LockFreeLinkedListNode { public inline fun <reified T : LockFreeLinkedListNode> forEach(block: (T) -> Unit) public final override fun remove(): Nothing } - -/** @suppress **This is unstable API and it is subject to change.** */ -public expect open class AddLastDesc<T : LockFreeLinkedListNode>( - queue: LockFreeLinkedListNode, - node: T -) : AbstractAtomicDesc { - val queue: LockFreeLinkedListNode - val node: T - override fun finishPrepare(prepareOp: PrepareOp) - override fun finishOnSuccess(affected: LockFreeLinkedListNode, next: LockFreeLinkedListNode) -} - -/** @suppress **This is unstable API and it is subject to change.** */ -public expect open class RemoveFirstDesc<T>(queue: LockFreeLinkedListNode): AbstractAtomicDesc { - val queue: LockFreeLinkedListNode - public val result: T - override fun finishPrepare(prepareOp: PrepareOp) - final override fun finishOnSuccess(affected: LockFreeLinkedListNode, next: LockFreeLinkedListNode) -} - -/** @suppress **This is unstable API and it is subject to change.** */ -public expect abstract class AbstractAtomicDesc : AtomicDesc { - final override fun prepare(op: AtomicOp<*>): Any? - final override fun complete(op: AtomicOp<*>, failure: Any?) - protected open fun failure(affected: LockFreeLinkedListNode): Any? - protected open fun retry(affected: LockFreeLinkedListNode, next: Any): Boolean - public abstract fun finishPrepare(prepareOp: PrepareOp) // non-null on failure - public open fun onPrepare(prepareOp: PrepareOp): Any? // non-null on failure - public open fun onRemoved(affected: LockFreeLinkedListNode) // non-null on failure - protected abstract fun finishOnSuccess(affected: LockFreeLinkedListNode, next: LockFreeLinkedListNode) -} - -/** @suppress **This is unstable API and it is subject to change.** */ -public expect class PrepareOp: OpDescriptor { - val affected: LockFreeLinkedListNode - override val atomicOp: AtomicOp<*> - val desc: AbstractAtomicDesc - fun finishPrepare() -} - -@JvmField -@SharedImmutable -internal val REMOVE_PREPARED: Any = Symbol("REMOVE_PREPARED") diff --git a/kotlinx-coroutines-core/common/src/internal/Scopes.kt b/kotlinx-coroutines-core/common/src/internal/Scopes.kt index ad8d86ed..e0a18302 100644 --- a/kotlinx-coroutines-core/common/src/internal/Scopes.kt +++ b/kotlinx-coroutines-core/common/src/internal/Scopes.kt @@ -21,7 +21,6 @@ internal open class ScopeCoroutine<in T>( final override fun getStackTraceElement(): StackTraceElement? = null final override val isScopedCoroutine: Boolean get() = true - internal val parent: Job? get() = parentHandle?.parent override fun afterCompletion(state: Any?) { // Resume in a cancellable way by default when resuming from another context diff --git a/kotlinx-coroutines-core/common/src/internal/Synchronized.common.kt b/kotlinx-coroutines-core/common/src/internal/Synchronized.common.kt index 059b234d..7107945a 100644 --- a/kotlinx-coroutines-core/common/src/internal/Synchronized.common.kt +++ b/kotlinx-coroutines-core/common/src/internal/Synchronized.common.kt @@ -5,6 +5,7 @@ package kotlinx.coroutines.internal import kotlinx.coroutines.* +import kotlin.contracts.* /** * @suppress **This an internal API and should not be used from general code.** @@ -16,4 +17,16 @@ public expect open class SynchronizedObject() // marker abstract class * @suppress **This an internal API and should not be used from general code.** */ @InternalCoroutinesApi -public expect inline fun <T> synchronized(lock: SynchronizedObject, block: () -> T): T +public expect inline fun <T> synchronizedImpl(lock: SynchronizedObject, block: () -> T): T + +/** + * @suppress **This an internal API and should not be used from general code.** + */ +@OptIn(ExperimentalContracts::class) +@InternalCoroutinesApi +public inline fun <T> synchronized(lock: SynchronizedObject, block: () -> T): T { + contract { + callsInPlace(block, InvocationKind.EXACTLY_ONCE) + } + return synchronizedImpl(lock, block) +} diff --git a/kotlinx-coroutines-core/common/src/internal/SystemProps.common.kt b/kotlinx-coroutines-core/common/src/internal/SystemProps.common.kt index ca84809b..281c075b 100644 --- a/kotlinx-coroutines-core/common/src/internal/SystemProps.common.kt +++ b/kotlinx-coroutines-core/common/src/internal/SystemProps.common.kt @@ -58,6 +58,17 @@ internal fun systemProp( /** * Gets the system property indicated by the specified [property name][propertyName], + * or returns [defaultValue] if there is no property with that key. + * + * **Note: this function should be used in JVM tests only, other platforms use the default value.** + */ +internal fun systemProp( + propertyName: String, + defaultValue: String +): String = systemProp(propertyName) ?: defaultValue + +/** + * Gets the system property indicated by the specified [property name][propertyName], * or returns `null` if there is no property with that key. * * **Note: this function should be used in JVM tests only, other platforms use the default value.** diff --git a/kotlinx-coroutines-core/common/src/internal/ThreadLocal.common.kt b/kotlinx-coroutines-core/common/src/internal/ThreadLocal.common.kt index 890ae4e3..73ec93f1 100644 --- a/kotlinx-coroutines-core/common/src/internal/ThreadLocal.common.kt +++ b/kotlinx-coroutines-core/common/src/internal/ThreadLocal.common.kt @@ -4,7 +4,16 @@ package kotlinx.coroutines.internal -internal expect class CommonThreadLocal<T>() { +internal expect class CommonThreadLocal<T> { fun get(): T fun set(value: T) } + +/** + * Create a thread-local storage for an object of type [T]. + * + * If two different thread-local objects share the same [name], they will not necessarily share the same value, + * but they may. + * Therefore, use a unique [name] for each thread-local object. + */ +internal expect fun<T> commonThreadLocal(name: Symbol): CommonThreadLocal<T> diff --git a/kotlinx-coroutines-core/common/src/intrinsics/Undispatched.kt b/kotlinx-coroutines-core/common/src/intrinsics/Undispatched.kt index 38e870ef..3fa53c4b 100644 --- a/kotlinx-coroutines-core/common/src/intrinsics/Undispatched.kt +++ b/kotlinx-coroutines-core/common/src/intrinsics/Undispatched.kt @@ -21,17 +21,6 @@ internal fun <T> (suspend () -> T).startCoroutineUnintercepted(completion: Conti } /** - * Use this function to restart a coroutine directly from inside of [suspendCoroutine], - * when the code is already in the context of this coroutine. - * It does not use [ContinuationInterceptor] and does not update the context of the current thread. - */ -internal fun <R, T> (suspend (R) -> T).startCoroutineUnintercepted(receiver: R, completion: Continuation<T>) { - startDirect(completion) { actualCompletion -> - startCoroutineUninterceptedOrReturn(receiver, actualCompletion) - } -} - -/** * Use this function to start a new coroutine in [CoroutineStart.UNDISPATCHED] mode — * immediately execute the coroutine in the current thread until the next suspension. * It does not use [ContinuationInterceptor], but updates the context of the current thread for the new coroutine. diff --git a/kotlinx-coroutines-core/common/src/selects/OnTimeout.kt b/kotlinx-coroutines-core/common/src/selects/OnTimeout.kt new file mode 100644 index 00000000..ba391813 --- /dev/null +++ b/kotlinx-coroutines-core/common/src/selects/OnTimeout.kt @@ -0,0 +1,65 @@ +/* + * Copyright 2016-2021 JetBrains s.r.o. Use of this source code is governed by the Apache 2.0 license. + */ + +package kotlinx.coroutines.selects + +import kotlinx.coroutines.* +import kotlin.time.* + +/** + * Clause that selects the given [block] after a specified timeout passes. + * If timeout is negative or zero, [block] is selected immediately. + * + * **Note: This is an experimental api.** It may be replaced with light-weight timer/timeout channels in the future. + * + * @param timeMillis timeout time in milliseconds. + */ +@ExperimentalCoroutinesApi +@Suppress("EXTENSION_SHADOWED_BY_MEMBER") +public fun <R> SelectBuilder<R>.onTimeout(timeMillis: Long, block: suspend () -> R): Unit = + OnTimeout(timeMillis).selectClause.invoke(block) + +/** + * Clause that selects the given [block] after the specified [timeout] passes. + * If timeout is negative or zero, [block] is selected immediately. + * + * **Note: This is an experimental api.** It may be replaced with light-weight timer/timeout channels in the future. + */ +@ExperimentalCoroutinesApi +public fun <R> SelectBuilder<R>.onTimeout(timeout: Duration, block: suspend () -> R): Unit = + onTimeout(timeout.toDelayMillis(), block) + +/** + * We implement [SelectBuilder.onTimeout] as a clause, so each invocation creates + * an instance of [OnTimeout] that specifies the registration part according to + * the [timeout][timeMillis] parameter. + */ +private class OnTimeout( + private val timeMillis: Long +) { + @Suppress("UNCHECKED_CAST") + val selectClause: SelectClause0 + get() = SelectClause0Impl( + clauseObject = this@OnTimeout, + regFunc = OnTimeout::register as RegistrationFunction + ) + + @Suppress("UNUSED_PARAMETER") + private fun register(select: SelectInstance<*>, ignoredParam: Any?) { + // Should this clause complete immediately? + if (timeMillis <= 0) { + select.selectInRegistrationPhase(Unit) + return + } + // Invoke `trySelect` after the timeout is reached. + val action = Runnable { + select.trySelect(this@OnTimeout, Unit) + } + select as SelectImplementation<*> + val context = select.context + val disposableHandle = context.delay.invokeOnTimeout(timeMillis, action, context) + // Do not forget to clean-up when this `select` is completed or cancelled. + select.disposeOnCompletion(disposableHandle) + } +} diff --git a/kotlinx-coroutines-core/common/src/selects/Select.kt b/kotlinx-coroutines-core/common/src/selects/Select.kt index 92132224..3ac3cb6f 100644 --- a/kotlinx-coroutines-core/common/src/selects/Select.kt +++ b/kotlinx-coroutines-core/common/src/selects/Select.kt @@ -1,7 +1,6 @@ /* * Copyright 2016-2021 JetBrains s.r.o. Use of this source code is governed by the Apache 2.0 license. */ -@file:OptIn(ExperimentalContracts::class) package kotlinx.coroutines.selects @@ -9,19 +8,73 @@ import kotlinx.atomicfu.* import kotlinx.coroutines.* import kotlinx.coroutines.channels.* import kotlinx.coroutines.internal.* -import kotlinx.coroutines.intrinsics.* -import kotlinx.coroutines.sync.* +import kotlinx.coroutines.selects.TrySelectDetailedResult.* import kotlin.contracts.* import kotlin.coroutines.* -import kotlin.coroutines.intrinsics.* +import kotlin.internal.* import kotlin.jvm.* -import kotlin.native.concurrent.* -import kotlin.time.* + +/** + * Waits for the result of multiple suspending functions simultaneously, which are specified using _clauses_ + * in the [builder] scope of this select invocation. The caller is suspended until one of the clauses + * is either _selected_ or _fails_. + * + * At most one clause is *atomically* selected and its block is executed. The result of the selected clause + * becomes the result of the select. If any clause _fails_, then the select invocation produces the + * corresponding exception. No clause is selected in this case. + * + * This select function is _biased_ to the first clause. When multiple clauses can be selected at the same time, + * the first one of them gets priority. Use [selectUnbiased] for an unbiased (randomized) selection among + * the clauses. + + * There is no `default` clause for select expression. Instead, each selectable suspending function has the + * corresponding non-suspending version that can be used with a regular `when` expression to select one + * of the alternatives or to perform the default (`else`) action if none of them can be immediately selected. + * + * ### List of supported select methods + * + * | **Receiver** | **Suspending function** | **Select clause** + * | ---------------- | --------------------------------------------- | ----------------------------------------------------- + * | [Job] | [join][Job.join] | [onJoin][Job.onJoin] + * | [Deferred] | [await][Deferred.await] | [onAwait][Deferred.onAwait] + * | [SendChannel] | [send][SendChannel.send] | [onSend][SendChannel.onSend] + * | [ReceiveChannel] | [receive][ReceiveChannel.receive] | [onReceive][ReceiveChannel.onReceive] + * | [ReceiveChannel] | [receiveCatching][ReceiveChannel.receiveCatching] | [onReceiveCatching][ReceiveChannel.onReceiveCatching] + * | none | [delay] | [onTimeout][SelectBuilder.onTimeout] + * + * This suspending function is cancellable. If the [Job] of the current coroutine is cancelled or completed while this + * function is suspended, this function immediately resumes with [CancellationException]. + * There is a **prompt cancellation guarantee**. If the job was cancelled while this function was + * suspended, it will not resume successfully. See [suspendCancellableCoroutine] documentation for low-level details. + * + * Note that this function does not check for cancellation when it is not suspended. + * Use [yield] or [CoroutineScope.isActive] to periodically check for cancellation in tight loops if needed. + */ +@OptIn(ExperimentalContracts::class) +public suspend inline fun <R> select(crossinline builder: SelectBuilder<R>.() -> Unit): R { + contract { + callsInPlace(builder, InvocationKind.EXACTLY_ONCE) + } + return SelectImplementation<R>(coroutineContext).run { + builder(this) + // TAIL-CALL OPTIMIZATION: the only + // suspend call is at the last position. + doSelect() + } +} /** * Scope for [select] invocation. + * + * An instance of [SelectBuilder] can only be retrieved as a receiver of a [select] block call, + * and it is only valid during the registration phase of the select builder. + * Any uses outside it lead to unspecified behaviour and are prohibited. + * + * The general rule of thumb is that instances of this type should always be used + * implicitly and there shouldn't be any signatures mentioning this type, + * whether explicitly (e.g. function signature) or implicitly (e.g. inferred `val` type). */ -public interface SelectBuilder<in R> { +public sealed interface SelectBuilder<in R> { /** * Registers a clause in this [select] expression without additional parameters that does not select any value. */ @@ -52,606 +105,768 @@ public interface SelectBuilder<in R> { * @param timeMillis timeout time in milliseconds. */ @ExperimentalCoroutinesApi - public fun onTimeout(timeMillis: Long, block: suspend () -> R) + @Suppress("INVISIBLE_REFERENCE", "INVISIBLE_MEMBER") + @LowPriorityInOverloadResolution + @Deprecated( + message = "Replaced with the same extension function", + level = DeprecationLevel.ERROR, replaceWith = ReplaceWith(expression = "onTimeout", imports = ["kotlinx.coroutines.selects.onTimeout"]) + ) // Since 1.7.0, was experimental + public fun onTimeout(timeMillis: Long, block: suspend () -> R): Unit = onTimeout(timeMillis, block) } /** - * Clause that selects the given [block] after the specified [timeout] passes. - * If timeout is negative or zero, [block] is selected immediately. - * - * **Note: This is an experimental api.** It may be replaced with light-weight timer/timeout channels in the future. + * Each [select] clause is specified with: + * 1) the [object of this clause][clauseObject], + * such as the channel instance for [SendChannel.onSend]; + * 2) the function that specifies how this clause + * should be registered in the object above; + * 3) the function that modifies the internal result + * (passed via [SelectInstance.trySelect] or + * [SelectInstance.selectInRegistrationPhase]) + * to the argument of the user-specified block. + * 4) the function that specifies how the internal result provided via + * [SelectInstance.trySelect] or [SelectInstance.selectInRegistrationPhase] + * should be processed in case of this `select` cancellation while dispatching. + */ +@InternalCoroutinesApi +public sealed interface SelectClause { + public val clauseObject: Any + public val regFunc: RegistrationFunction + public val processResFunc: ProcessResultFunction + public val onCancellationConstructor: OnCancellationConstructor? +} + +/** + * The registration function specifies how the `select` instance should be registered into + * the specified clause object. In case of channels, the registration logic + * coincides with the plain `send/receive` operation with the only difference that + * the `select` instance is stored as a waiter instead of continuation. + */ +@InternalCoroutinesApi +public typealias RegistrationFunction = (clauseObject: Any, select: SelectInstance<*>, param: Any?) -> Unit + +/** + * This function specifies how the _internal_ result, provided via [SelectInstance.selectInRegistrationPhase] + * or [SelectInstance.trySelect] should be processed. For example, both [ReceiveChannel.onReceive] and + * [ReceiveChannel.onReceiveCatching] clauses perform exactly the same synchronization logic, + * but differ when the channel has been discovered in the closed or cancelled state. */ -@ExperimentalCoroutinesApi -public fun <R> SelectBuilder<R>.onTimeout(timeout: Duration, block: suspend () -> R): Unit = - onTimeout(timeout.toDelayMillis(), block) +@InternalCoroutinesApi +public typealias ProcessResultFunction = (clauseObject: Any, param: Any?, clauseResult: Any?) -> Any? + +/** + * This function specifies how the internal result, provided via [SelectInstance.trySelect] + * or [SelectInstance.selectInRegistrationPhase], should be processed in case of this `select` + * cancellation while dispatching. Unfortunately, we cannot pass this function only in [SelectInstance.trySelect], + * as [SelectInstance.selectInRegistrationPhase] can be called when the coroutine is already cancelled. + */ +@InternalCoroutinesApi +public typealias OnCancellationConstructor = (select: SelectInstance<*>, param: Any?, internalResult: Any?) -> (Throwable) -> Unit /** * Clause for [select] expression without additional parameters that does not select any value. */ -public interface SelectClause0 { - /** - * Registers this clause with the specified [select] instance and [block] of code. - * @suppress **This is unstable API and it is subject to change.** - */ - @InternalCoroutinesApi - public fun <R> registerSelectClause0(select: SelectInstance<R>, block: suspend () -> R) +public sealed interface SelectClause0 : SelectClause + +internal class SelectClause0Impl( + override val clauseObject: Any, + override val regFunc: RegistrationFunction, + override val onCancellationConstructor: OnCancellationConstructor? = null +) : SelectClause0 { + override val processResFunc: ProcessResultFunction = DUMMY_PROCESS_RESULT_FUNCTION } +private val DUMMY_PROCESS_RESULT_FUNCTION: ProcessResultFunction = { _, _, _ -> null } /** * Clause for [select] expression without additional parameters that selects value of type [Q]. */ -public interface SelectClause1<out Q> { - /** - * Registers this clause with the specified [select] instance and [block] of code. - * @suppress **This is unstable API and it is subject to change.** - */ - @InternalCoroutinesApi - public fun <R> registerSelectClause1(select: SelectInstance<R>, block: suspend (Q) -> R) -} +public sealed interface SelectClause1<out Q> : SelectClause + +internal class SelectClause1Impl<Q>( + override val clauseObject: Any, + override val regFunc: RegistrationFunction, + override val processResFunc: ProcessResultFunction, + override val onCancellationConstructor: OnCancellationConstructor? = null +) : SelectClause1<Q> /** * Clause for [select] expression with additional parameter of type [P] that selects value of type [Q]. */ -public interface SelectClause2<in P, out Q> { - /** - * Registers this clause with the specified [select] instance and [block] of code. - * @suppress **This is unstable API and it is subject to change.** - */ - @InternalCoroutinesApi - public fun <R> registerSelectClause2(select: SelectInstance<R>, param: P, block: suspend (Q) -> R) -} +public sealed interface SelectClause2<in P, out Q> : SelectClause + +internal class SelectClause2Impl<P, Q>( + override val clauseObject: Any, + override val regFunc: RegistrationFunction, + override val processResFunc: ProcessResultFunction, + override val onCancellationConstructor: OnCancellationConstructor? = null +) : SelectClause2<P, Q> /** - * Internal representation of select instance. This instance is called _selected_ when - * the clause to execute is already picked. + * Internal representation of `select` instance. * - * @suppress **This is unstable API and it is subject to change.** + * @suppress **This is unstable API, and it is subject to change.** */ -@InternalCoroutinesApi // todo: sealed interface https://youtrack.jetbrains.com/issue/KT-22286 -public interface SelectInstance<in R> { +@InternalCoroutinesApi +public sealed interface SelectInstance<in R> { /** - * Returns `true` when this [select] statement had already picked a clause to execute. + * The context of the coroutine that is performing this `select` operation. */ - public val isSelected: Boolean + public val context: CoroutineContext /** - * Tries to select this instance. Returns `true` on success. + * This function should be called by other operations, + * which are trying to perform a rendezvous with this `select`. + * Returns `true` if the rendezvous succeeds, `false` otherwise. + * + * Note that according to the current implementation, a rendezvous attempt can fail + * when either another clause is already selected or this `select` is still in + * REGISTRATION phase. To distinguish the reasons, [SelectImplementation.trySelectDetailed] + * function can be used instead. */ - public fun trySelect(): Boolean + public fun trySelect(clauseObject: Any, result: Any?): Boolean /** - * Tries to select this instance. Returns: - * * [RESUME_TOKEN] on success, - * * [RETRY_ATOMIC] on deadlock (needs retry, it is only possible when [otherOp] is not `null`) - * * `null` on failure to select (already selected). - * [otherOp] is not null when trying to rendezvous with this select from inside of another select. - * In this case, [PrepareOp.finishPrepare] must be called before deciding on any value other than [RETRY_ATOMIC]. - * - * Note, that this method's actual return type is `Symbol?` but we cannot declare it as such, because this - * member is public, but [Symbol] is internal. When [SelectInstance] becomes a `sealed interface` - * (see KT-222860) we can declare this method as internal. + * When this `select` instance is stored as a waiter, the specified [handle][disposableHandle] + * defines how the stored `select` should be removed in case of cancellation or another clause selection. */ - public fun trySelectOther(otherOp: PrepareOp?): Any? + public fun disposeOnCompletion(disposableHandle: DisposableHandle) /** - * Performs action atomically with [trySelect]. - * May return [RETRY_ATOMIC], caller shall retry with **fresh instance of desc**. + * When a clause becomes selected during registration, the corresponding internal result + * (which is further passed to the clause's [ProcessResultFunction]) should be provided + * via this function. After that, other clause registrations are ignored and [trySelect] fails. */ - public fun performAtomicTrySelect(desc: AtomicDesc): Any? + public fun selectInRegistrationPhase(internalResult: Any?) +} +internal interface SelectInstanceInternal<R>: SelectInstance<R>, Waiter + +@PublishedApi +internal open class SelectImplementation<R>( + override val context: CoroutineContext +) : CancelHandler(), SelectBuilder<R>, SelectInstanceInternal<R> { /** - * Returns completion continuation of this select instance. - * This select instance must be _selected_ first. - * All resumption through this instance happen _directly_ without going through dispatcher. + * Essentially, the `select` operation is split into three phases: REGISTRATION, WAITING, and COMPLETION. + * + * == Phase 1: REGISTRATION == + * In the first REGISTRATION phase, the user-specified [SelectBuilder] is applied, and all the listed clauses + * are registered via the provided [registration functions][SelectClause.regFunc]. Intuitively, `select` clause + * registration is similar to the plain blocking operation, with the only difference that this [SelectInstance] + * is stored as a waiter instead of continuation, and [SelectInstance.trySelect] is used to make a rendezvous. + * Also, when registering, it is possible for the operation to complete immediately, without waiting. In this case, + * [SelectInstance.selectInRegistrationPhase] should be used. Otherwise, when no rendezvous happens and this `select` + * instance is stored as a waiter, a completion handler for the registering clause should be specified via + * [SelectInstance.disposeOnCompletion]; this handler specifies how to remove this `select` instance from the + * clause object when another clause becomes selected or the operation cancels. + * + * After a clause registration is completed, another coroutine can attempt to make a rendezvous with this `select`. + * However, to resolve a race between clauses registration and [SelectInstance.trySelect], the latter fails when + * this `select` is still in REGISTRATION phase. Thus, the corresponding clause has to be registered again. + * + * In this phase, the `state` field stores either a special [STATE_REG] marker or + * a list of clauses to be re-registered due to failed rendezvous attempts. + * + * == Phase 2: WAITING == + * If no rendezvous happens in REGISTRATION phase, the `select` operation moves to WAITING one and suspends until + * [SelectInstance.trySelect] is called. Also, when waiting, this `select` can be cancelled. In the latter case, + * further [SelectInstance.trySelect] attempts fail, and all the completion handlers, specified via + * [SelectInstance.disposeOnCompletion], are invoked to remove this `select` instance from the corresponding + * clause objects. + * + * In this phase, the `state` field stores either the continuation to be later resumed or a special `Cancelled` + * object (with the cancellation cause inside) when this `select` becomes cancelled. + * + * == Phase 3: COMPLETION == + * Once a rendezvous happens either in REGISTRATION phase (via [SelectInstance.selectInRegistrationPhase]) or + * in WAITING phase (via [SelectInstance.trySelect]), this `select` moves to the final `COMPLETION` phase. + * First, the provided internal result is processed via the [ProcessResultFunction] of the selected clause; + * it returns the argument for the user-specified block or throws an exception (see [SendChannel.onSend] as + * an example). After that, this `select` should be removed from all other clause objects by calling the + * corresponding [DisposableHandle]-s, provided via [SelectInstance.disposeOnCompletion] during registration. + * At the end, the user-specified block is called and this `select` finishes. + * + * In this phase, once a rendezvous is happened, the `state` field stores the corresponding clause. + * After that, it moves to [STATE_COMPLETED] to avoid memory leaks. + * + * + * + * The state machine is listed below: + * + * REGISTRATION PHASE WAITING PHASE COMPLETION PHASE + * ⌢⌢⌢⌢⌢⌢⌢⌢⌢⌢⌢⌢⌢⌢⌢⌢⌢⌢⌢⌢ ⌢⌢⌢⌢⌢⌢⌢⌢⌢⌢⌢⌢ ⌢⌢⌢⌢⌢⌢⌢⌢⌢⌢⌢⌢⌢⌢ + * + * +-----------+ +-----------+ + * | CANCELLED | | COMPLETED | + * +-----------+ +-----------+ + * ^ ^ + * INITIAL STATE | | this `select` + * ------------+ | cancelled | is completed + * \ | | + * +=============+ move to +------+ successful +------------+ + * +--| STATE_REG |---------------> | cont |-----------------| ClauseData | + * | +=============+ WAITING phase +------+ trySelect(..) +------------+ + * | ^ | ^ + * | | | some clause has been selected during registration | + * add a | | +-------------------------------------------------------+ + * clause to be | | | + * re-registered | | re-register some clause has been selected | + * | | clauses during registration while there | + * v | are clauses to be re-registered; | + * +------------------+ ignore the latter | + * +--| List<ClauseData> |-----------------------------------------------------+ + * | +------------------+ + * | ^ + * | | add one more clause + * | | for re-registration + * +------------+ + * + * One of the most valuable benefits of this `select` design is that it allows processing clauses + * in a way similar to plain operations, such as `send` or `receive` on channels. The only difference + * is that instead of continuation, the operation should store the provided `select` instance object. + * Thus, this design makes it possible to support the `select` expression for any blocking data structure + * in Kotlin Coroutines. + * + * It is worth mentioning that the algorithm above provides "obstruction-freedom" non-blocking guarantee + * instead of the standard "lock-freedom" to avoid using heavy descriptors. In practice, this relaxation + * does not make significant difference. However, it is vital for Kotlin Coroutines to provide some + * non-blocking guarantee, as users may add blocking code in [SelectBuilder], and this blocking code + * should not cause blocking behaviour in other places, such as an attempt to make a rendezvous with + * the `select` that is hang in REGISTRATION phase. + * + * Also, this implementation is NOT linearizable under some circumstances. The reason is that a rendezvous + * attempt with `select` (via [SelectInstance.trySelect]) may fail when this `select` operation is still + * in REGISTRATION phase. Consider the following situation on two empty rendezvous channels `c1` and `c2` + * and the `select` operation that tries to send an element to one of these channels. First, this `select` + * instance is registered as a waiter in `c1`. After that, another thread can observe that `c1` is no longer + * empty and try to receive an element from `c1` -- this receive attempt fails due to the `select` operation + * being in REGISTRATION phase. + * It is also possible to observe that this `select` operation registered in `c2` first, and only after that in + * `c1` (it has to re-register in `c1` after the unsuccessful rendezvous attempt), which is also non-linearizable. + * We, however, find such a non-linearizable behaviour not so important in practice and leverage the correctness + * relaxation for the algorithm simplicity and the non-blocking progress guarantee. */ - public val completion: Continuation<R> /** - * Resumes this instance in a dispatched way with exception. - * This method can be called from any context. + * The state of this `select` operation. See the description above for details. */ - public fun resumeSelectWithException(exception: Throwable) - + private val state = atomic<Any>(STATE_REG) /** - * Disposes the specified handle when this instance is selected. - * Note, that [DisposableHandle.dispose] could be called multiple times. + * Returns `true` if this `select` instance is in the REGISTRATION phase; + * otherwise, returns `false`. */ - public fun disposeOnSelect(handle: DisposableHandle) -} - -/** - * Waits for the result of multiple suspending functions simultaneously, which are specified using _clauses_ - * in the [builder] scope of this select invocation. The caller is suspended until one of the clauses - * is either _selected_ or _fails_. - * - * At most one clause is *atomically* selected and its block is executed. The result of the selected clause - * becomes the result of the select. If any clause _fails_, then the select invocation produces the - * corresponding exception. No clause is selected in this case. - * - * This select function is _biased_ to the first clause. When multiple clauses can be selected at the same time, - * the first one of them gets priority. Use [selectUnbiased] for an unbiased (randomized) selection among - * the clauses. - - * There is no `default` clause for select expression. Instead, each selectable suspending function has the - * corresponding non-suspending version that can be used with a regular `when` expression to select one - * of the alternatives or to perform the default (`else`) action if none of them can be immediately selected. - * - * ### List of supported select methods - * - * | **Receiver** | **Suspending function** | **Select clause** - * | ---------------- | --------------------------------------------- | ----------------------------------------------------- - * | [Job] | [join][Job.join] | [onJoin][Job.onJoin] - * | [Deferred] | [await][Deferred.await] | [onAwait][Deferred.onAwait] - * | [SendChannel] | [send][SendChannel.send] | [onSend][SendChannel.onSend] - * | [ReceiveChannel] | [receive][ReceiveChannel.receive] | [onReceive][ReceiveChannel.onReceive] - * | [ReceiveChannel] | [receiveCatching][ReceiveChannel.receiveCatching] | [onReceiveCatching][ReceiveChannel.onReceiveCatching] - * | none | [delay] | [onTimeout][SelectBuilder.onTimeout] - * - * This suspending function is cancellable. If the [Job] of the current coroutine is cancelled or completed while this - * function is suspended, this function immediately resumes with [CancellationException]. - * There is a **prompt cancellation guarantee**. If the job was cancelled while this function was - * suspended, it will not resume successfully. See [suspendCancellableCoroutine] documentation for low-level details. - * - * Note that this function does not check for cancellation when it is not suspended. - * Use [yield] or [CoroutineScope.isActive] to periodically check for cancellation in tight loops if needed. - */ -public suspend inline fun <R> select(crossinline builder: SelectBuilder<R>.() -> Unit): R { - contract { - callsInPlace(builder, InvocationKind.EXACTLY_ONCE) - } - return suspendCoroutineUninterceptedOrReturn { uCont -> - val scope = SelectBuilderImpl(uCont) - try { - builder(scope) - } catch (e: Throwable) { - scope.handleBuilderException(e) + private val inRegistrationPhase + get() = state.value.let { + it === STATE_REG || it is List<*> } - scope.getResult() - } -} + /** + * Returns `true` if this `select` is already selected; + * thus, other parties are bound to fail when making a rendezvous with it. + */ + private val isSelected + get() = state.value is SelectImplementation<*>.ClauseData + /** + * Returns `true` if this `select` is cancelled. + */ + private val isCancelled + get() = state.value === STATE_CANCELLED + /** + * List of clauses waiting on this `select` instance. + */ + private var clauses: MutableList<ClauseData>? = ArrayList(2) -@SharedImmutable -internal val NOT_SELECTED: Any = Symbol("NOT_SELECTED") -@SharedImmutable -internal val ALREADY_SELECTED: Any = Symbol("ALREADY_SELECTED") -@SharedImmutable -private val UNDECIDED: Any = Symbol("UNDECIDED") -@SharedImmutable -private val RESUMED: Any = Symbol("RESUMED") - -// Global counter of all atomic select operations for their deadlock resolution -// The separate internal class is work-around for Atomicfu's current implementation that creates public classes -// for static atomics -internal class SeqNumber { - private val number = atomic(1L) - fun next() = number.incrementAndGet() -} + /** + * Stores the completion action provided through [disposeOnCompletion] or [invokeOnCancellation] + * during clause registration. After that, if the clause is successfully registered + * (so, it has not completed immediately), this handler is stored into + * the corresponding [ClauseData] instance. + * + * Note that either [DisposableHandle] is provided, or a [Segment] instance with + * the index in it, which specify the location of storing this `select`. + * In the latter case, [Segment.onCancellation] should be called on completion/cancellation. + */ + private var disposableHandleOrSegment: Any? = null -@SharedImmutable -private val selectOpSequenceNumber = SeqNumber() + /** + * In case the disposable handle is specified via [Segment] + * and index in it, implying calling [Segment.onCancellation], + * the corresponding index is stored in this field. + * The segment is stored in [disposableHandleOrSegment]. + */ + private var indexInSegment: Int = -1 -@PublishedApi -internal class SelectBuilderImpl<in R>( - private val uCont: Continuation<R> // unintercepted delegate continuation -) : LockFreeLinkedListHead(), SelectBuilder<R>, - SelectInstance<R>, Continuation<R>, CoroutineStackFrame -{ - override val callerFrame: CoroutineStackFrame? - get() = uCont as? CoroutineStackFrame - - override fun getStackTraceElement(): StackTraceElement? = null - - // selection state is NOT_SELECTED initially and is replaced by idempotent marker (or null) when selected - private val _state = atomic<Any?>(NOT_SELECTED) - - // this is basically our own SafeContinuation - private val _result = atomic<Any?>(UNDECIDED) - - // cancellability support - private val _parentHandle = atomic<DisposableHandle?>(null) - private var parentHandle: DisposableHandle? - get() = _parentHandle.value - set(value) { _parentHandle.value = value } - - /* Result state machine - - +-----------+ getResult +---------------------+ resume +---------+ - | UNDECIDED | ------------> | COROUTINE_SUSPENDED | ---------> | RESUMED | - +-----------+ +---------------------+ +---------+ - | - | resume - V - +------------+ getResult - | value/Fail | -----------+ - +------------+ | - ^ | - | | - +-------------------+ + /** + * Stores the result passed via [selectInRegistrationPhase] during clause registration + * or [trySelect], which is called by another coroutine trying to make a rendezvous + * with this `select` instance. Further, this result is processed via the + * [ProcessResultFunction] of the selected clause. + * + * Unfortunately, we cannot store the result in the [state] field, as the latter stores + * the clause object upon selection (see [ClauseData.clauseObject] and [SelectClause.clauseObject]). + * Instead, it is possible to merge the [internalResult] and [disposableHandle] fields into + * one that stores either result when the clause is successfully registered ([inRegistrationPhase] is `true`), + * or [DisposableHandle] instance when the clause is completed during registration ([inRegistrationPhase] is `false`). + * Yet, this optimization is omitted for code simplicity. */ + private var internalResult: Any? = NO_RESULT - override val context: CoroutineContext get() = uCont.context + /** + * This function is called after the [SelectBuilder] is applied. In case one of the clauses is already selected, + * the algorithm applies the corresponding [ProcessResultFunction] and invokes the user-specified [block][ClauseData.block]. + * Otherwise, it moves this `select` to WAITING phase (re-registering clauses if needed), suspends until a rendezvous + * is happened, and then completes the operation by applying the corresponding [ProcessResultFunction] and + * invoking the user-specified [block][ClauseData.block]. + */ + @PublishedApi + internal open suspend fun doSelect(): R = + if (isSelected) complete() // Fast path + else doSelectSuspend() // Slow path + + // We separate the following logic as it has two suspension points + // and, therefore, breaks the tail-call optimization if it were + // inlined in [doSelect] + private suspend fun doSelectSuspend(): R { + // In case no clause has been selected during registration, + // the `select` operation suspends and waits for a rendezvous. + waitUntilSelected() // <-- suspend call => no tail-call optimization here + // There is a selected clause! Apply the corresponding + // [ProcessResultFunction] and invoke the user-specified block. + return complete() // <-- one more suspend call + } - override val completion: Continuation<R> get() = this + // ======================== + // = CLAUSES REGISTRATION = + // ======================== - private inline fun doResume(value: () -> Any?, block: () -> Unit) { - assert { isSelected } // "Must be selected first" - _result.loop { result -> - when { - result === UNDECIDED -> { - val update = value() - if (_result.compareAndSet(UNDECIDED, update)) return - } - result === COROUTINE_SUSPENDED -> if (_result.compareAndSet(COROUTINE_SUSPENDED, RESUMED)) { - block() - return - } - else -> throw IllegalStateException("Already resumed") - } - } - } + override fun SelectClause0.invoke(block: suspend () -> R) = + ClauseData(clauseObject, regFunc, processResFunc, PARAM_CLAUSE_0, block, onCancellationConstructor).register() + override fun <Q> SelectClause1<Q>.invoke(block: suspend (Q) -> R) = + ClauseData(clauseObject, regFunc, processResFunc, null, block, onCancellationConstructor).register() + override fun <P, Q> SelectClause2<P, Q>.invoke(param: P, block: suspend (Q) -> R) = + ClauseData(clauseObject, regFunc, processResFunc, param, block, onCancellationConstructor).register() - // Resumes in direct mode, without going through dispatcher. Should be called in the same context. - override fun resumeWith(result: Result<R>) { - doResume({ result.toState() }) { - if (result.isFailure) { - uCont.resumeWithStackTrace(result.exceptionOrNull()!!) - } else { - uCont.resumeWith(result) - } + /** + * Attempts to register this `select` clause. If another clause is already selected, + * this function does nothing and completes immediately. + * Otherwise, it registers this `select` instance in + * the [clause object][ClauseData.clauseObject] + * according to the provided [registration function][ClauseData.regFunc]. + * On success, this `select` instance is stored as a waiter + * in the clause object -- the algorithm also stores + * the provided via [disposeOnCompletion] completion action + * and adds the clause to the list of registered one. + * In case of registration failure, the internal result + * (not processed by [ProcessResultFunction] yet) must be + * provided via [selectInRegistrationPhase] -- the algorithm + * updates the state to this clause reference. + */ + @JvmName("register") + internal fun ClauseData.register(reregister: Boolean = false) { + assert { state.value !== STATE_CANCELLED } + // Is there already selected clause? + if (state.value.let { it is SelectImplementation<*>.ClauseData }) return + // For new clauses, check that there does not exist + // another clause with the same object. + if (!reregister) checkClauseObject(clauseObject) + // Try to register in the corresponding object. + if (tryRegisterAsWaiter(this@SelectImplementation)) { + // Successfully registered, and this `select` instance + // is stored as a waiter. Add this clause to the list + // of registered clauses and store the provided via + // [invokeOnCompletion] completion action into the clause. + // + // Importantly, the [waitUntilSelected] function is implemented + // carefully to ensure that the cancellation handler has not been + // installed when clauses re-register, so the logic below cannot + // be invoked concurrently with the clean-up procedure. + // This also guarantees that the list of clauses cannot be cleared + // in the registration phase, so it is safe to read it with "!!". + if (!reregister) clauses!! += this + disposableHandleOrSegment = this@SelectImplementation.disposableHandleOrSegment + indexInSegment = this@SelectImplementation.indexInSegment + this@SelectImplementation.disposableHandleOrSegment = null + this@SelectImplementation.indexInSegment = -1 + } else { + // This clause has been selected! + // Update the state correspondingly. + state.value = this } } - // Resumes in dispatched way so that it can be called from an arbitrary context - override fun resumeSelectWithException(exception: Throwable) { - doResume({ CompletedExceptionally(recoverStackTrace(exception, uCont)) }) { - uCont.intercepted().resumeWith(Result.failure(exception)) + /** + * Checks that there does not exist another clause with the same object. + */ + private fun checkClauseObject(clauseObject: Any) { + // Read the list of clauses, it is guaranteed that it is non-null. + // In fact, it can become `null` only in the clean-up phase, while + // this check can be called only in the registration one. + val clauses = clauses!! + // Check that there does not exist another clause with the same object. + check(clauses.none { it.clauseObject === clauseObject }) { + "Cannot use select clauses on the same object: $clauseObject" } } - @PublishedApi - internal fun getResult(): Any? { - if (!isSelected) initCancellability() - var result = _result.value // atomic read - if (result === UNDECIDED) { - if (_result.compareAndSet(UNDECIDED, COROUTINE_SUSPENDED)) return COROUTINE_SUSPENDED - result = _result.value // reread volatile var - } - when { - result === RESUMED -> throw IllegalStateException("Already resumed") - result is CompletedExceptionally -> throw result.cause - else -> return result // either COROUTINE_SUSPENDED or data - } + override fun disposeOnCompletion(disposableHandle: DisposableHandle) { + this.disposableHandleOrSegment = disposableHandle } - private fun initCancellability() { - val parent = context[Job] ?: return - val newRegistration = parent.invokeOnCompletion( - onCancelling = true, handler = SelectOnCancelling().asHandler) - parentHandle = newRegistration - // now check our state _after_ registering - if (isSelected) newRegistration.dispose() + /** + * An optimized version for the code below that does not allocate + * a cancellation handler object and efficiently stores the specified + * [segment] and [index]. + * + * ``` + * disposeOnCompletion { + * segment.onCancellation(index, null) + * } + * ``` + */ + override fun invokeOnCancellation(segment: Segment<*>, index: Int) { + this.disposableHandleOrSegment = segment + this.indexInSegment = index } - private inner class SelectOnCancelling : JobCancellingNode() { - // Note: may be invoked multiple times, but only the first trySelect succeeds anyway - override fun invoke(cause: Throwable?) { - if (trySelect()) - resumeSelectWithException(job.getCancellationException()) - } + override fun selectInRegistrationPhase(internalResult: Any?) { + this.internalResult = internalResult } - @PublishedApi - internal fun handleBuilderException(e: Throwable) { - if (trySelect()) { - resumeWithException(e) - } else if (e !is CancellationException) { - /* - * Cannot handle this exception -- builder was already resumed with a different exception, - * so treat it as "unhandled exception". But only if it is not the completion reason - * and it's not the cancellation. Otherwise, in the face of structured concurrency - * the same exception will be reported to the global exception handler. - */ - val result = getResult() - if (result !is CompletedExceptionally || unwrap(result.cause) !== unwrap(e)) { - handleCoroutineException(context, e) - } - } - } + // ========================= + // = WAITING FOR SELECTION = + // ========================= - override val isSelected: Boolean get() = _state.loop { state -> - when { - state === NOT_SELECTED -> return false - state is OpDescriptor -> state.perform(this) // help - else -> return true // already selected + /** + * Suspends and waits until some clause is selected. However, it is possible for a concurrent + * coroutine to invoke [trySelect] while this `select` is still in REGISTRATION phase. + * In this case, [trySelect] marks the corresponding select clause to be re-registered, and + * this function performs registration of such clauses. After that, it atomically stores + * the continuation into the [state] field if there is no more clause to be re-registered. + */ + private suspend fun waitUntilSelected() = suspendCancellableCoroutine<Unit> sc@ { cont -> + // Update the state. + state.loop { curState -> + when { + // This `select` is in REGISTRATION phase, and there is no clause to be re-registered. + // Perform a transition to WAITING phase by storing the current continuation. + curState === STATE_REG -> if (state.compareAndSet(curState, cont)) { + // Perform a clean-up in case of cancellation. + // + // Importantly, we MUST install the cancellation handler + // only when the algorithm is bound to suspend. Otherwise, + // a race with [tryRegister] is possible, and the provided + // via [disposeOnCompletion] cancellation action can be ignored. + // Also, we MUST guarantee that this dispose handle is _visible_ + // according to the memory model, and we CAN guarantee this when + // the state is updated. + cont.invokeOnCancellation(this.asHandler) + return@sc + } + // This `select` is in REGISTRATION phase, but there are clauses that has to be registered again. + // Perform the required registrations and try again. + curState is List<*> -> if (state.compareAndSet(curState, STATE_REG)) { + @Suppress("UNCHECKED_CAST") + curState as List<Any> + curState.forEach { reregisterClause(it) } + } + // This `select` operation became completed during clauses re-registration. + curState is SelectImplementation<*>.ClauseData -> { + cont.resume(Unit, curState.createOnCancellationAction(this, internalResult)) + return@sc + } + // This `select` cannot be in any other state. + else -> error("unexpected state: $curState") + } } } - override fun disposeOnSelect(handle: DisposableHandle) { - val node = DisposeNode(handle) - // check-add-check pattern is Ok here since handle.dispose() is safe to be called multiple times - if (!isSelected) { - addLast(node) // add handle to list - // double-check node after adding - if (!isSelected) return // all ok - still not selected - } - // already selected - handle.dispose() + /** + * Re-registers the clause with the specified + * [clause object][clauseObject] after unsuccessful + * [trySelect] of this clause while the `select` + * was still in REGISTRATION phase. + */ + private fun reregisterClause(clauseObject: Any) { + val clause = findClause(clauseObject)!! // it is guaranteed that the corresponding clause is presented + clause.disposableHandleOrSegment = null + clause.indexInSegment = -1 + clause.register(reregister = true) } - private fun doAfterSelect() { - parentHandle?.dispose() - forEach<DisposeNode> { - it.handle.dispose() - } - } + // ============== + // = RENDEZVOUS = + // ============== - override fun trySelect(): Boolean { - val result = trySelectOther(null) - return when { - result === RESUME_TOKEN -> true - result == null -> false - else -> error("Unexpected trySelectIdempotent result $result") - } - } + override fun trySelect(clauseObject: Any, result: Any?): Boolean = + trySelectInternal(clauseObject, result) == TRY_SELECT_SUCCESSFUL - /* - Diagram for rendezvous between two select operations: - - +---------+ +------------------------+ state(c) - | Channel | | SelectBuilderImpl(1) | -----------------------------------+ - +---------+ +------------------------+ | - | queue ^ | - V | select | - +---------+ next +------------------------+ next +--------------+ | - | LLHead | ------> | Send/ReceiveSelect(3) | -+----> | NextNode ... | | - +---------+ +------------------------+ | +--------------+ | - ^ ^ | next(b) ^ | - | affected | V | | - | +-----------------+ next | V - | | PrepareOp(6) | ----------+ +-----------------+ - | +-----------------+ <-------------------- | PairSelectOp(7) | - | | desc +-----------------+ - | V - | queue +----------------------+ - +------------------------- | TryPoll/OfferDesc(5) | - +----------------------+ - atomicOp | ^ - V | desc - +----------------------+ impl +---------------------+ - | SelectBuilderImpl(2) | <----- | AtomicSelectOp(4) | - +----------------------+ +---------------------+ - | state(a) ^ - | | - +----------------------------+ - - - 0. The first select operation SelectBuilderImpl(1) had already registered Send/ReceiveSelect(3) node - in the channel. - 1. The second select operation SelectBuilderImpl(2) is trying to rendezvous calling - performAtomicTrySelect(TryPoll/TryOfferDesc). - 2. A linked pair of AtomicSelectOp(4) and TryPoll/OfferDesc(5) is created to initiate this operation. - 3. AtomicSelectOp.prepareSelectOp installs a reference to AtomicSelectOp(4) in SelectBuilderImpl(2).state(a) - field. STARTING AT THIS MOMENT CONCURRENT HELPERS CAN DISCOVER AND TRY TO HELP PERFORM THIS OPERATION. - 4. Then TryPoll/OfferDesc.prepare discovers "affectedNode" for this operation as Send/ReceiveSelect(3) and - creates PrepareOp(6) that references it. It installs reference to PrepareOp(6) in Send/ReceiveSelect(3).next(b) - instead of its original next pointer that was stored in PrepareOp(6).next. - 5. PrepareOp(6).perform calls TryPoll/OfferDesc(5).onPrepare which validates that PrepareOp(6).affected node - is of the correct type and tries to secure ability to resume it by calling affected.tryResumeSend/Receive. - Note, that different PrepareOp instances can be repeatedly created for different candidate nodes. If node is - found to be be resumed/selected, then REMOVE_PREPARED result causes Send/ReceiveSelect(3).next change to - undone and new PrepareOp is created with a different candidate node. Different concurrent helpers may end up - creating different PrepareOp instances, so it is important that they ultimately come to consensus about - node on which perform operation upon. - 6. Send/ReceiveSelect(3).affected.tryResumeSend/Receive forwards this call to SelectBuilderImpl.trySelectOther, - passing it a reference to PrepareOp(6) as an indication of the other select instance rendezvous. - 7. SelectBuilderImpl.trySelectOther creates PairSelectOp(7) and installs it as SelectBuilderImpl(1).state(c) - to secure the state of the first builder and commit ability to make it selected for this operation. - 8. NOW THE RENDEZVOUS IS FULLY PREPARED via descriptors installed at - - SelectBuilderImpl(2).state(a) - - Send/ReceiveSelect(3).next(b) - - SelectBuilderImpl(1).state(c) - Any concurrent operation that is trying to access any of the select instances or the queue is going to help. - Any helper that helps AtomicSelectOp(4) calls TryPoll/OfferDesc(5).prepare which tries to determine - "affectedNode" but is bound to discover the same Send/ReceiveSelect(3) node that cannot become - non-first node until this operation completes (there are no insertions to the head of the queue!) - We have not yet decided to complete this operation, but we cannot ever decide to complete this operation - on any other node but Send/ReceiveSelect(3), so it is now safe to perform the next step. - 9. PairSelectOp(7).perform calls PrepareOp(6).finishPrepare which copies PrepareOp(6).affected and PrepareOp(6).next - to the corresponding TryPoll/OfferDesc(5) fields. - 10. PairSelectOp(7).perform calls AtomicSelect(4).decide to reach consensus on successful completion of this - operation. This consensus is important in light of dead-lock resolution algorithm, because a stale helper - could have stumbled upon a higher-numbered atomic operation and had decided to abort this atomic operation, - reaching decision on RETRY_ATOMIC status of it. We cannot proceed with completion in this case and must abort, - all objects including AtomicSelectOp(4) will be dropped, reverting all the three updated pointers to - their original values and atomic operation will retry from scratch. - 11. NOW WITH SUCCESSFUL UPDATE OF AtomicSelectOp(4).consensus to null THE RENDEZVOUS IS COMMITTED. The rest - of the code proceeds to update: - - SelectBuilderImpl(1).state to TryPoll/OfferDesc(5) so that late helpers would know that we have - already successfully completed rendezvous. - - Send/ReceiveSelect(3).next to Removed(next) so that this node becomes marked as removed. - - SelectBuilderImpl(2).state to null to mark this select instance as selected. - - Note, that very late helper may try to perform this AtomicSelectOp(4) when it is already completed. - It can proceed as far as finding affected node, creating PrepareOp, installing this new PrepareOp into the - node's next pointer, but PrepareOp.perform checks that AtomicSelectOp(4) is already decided and undoes all - the preparations. + /** + * Similar to [trySelect] but provides a failure reason + * if this rendezvous is unsuccessful. We need this function + * in the channel implementation. */ - - // it is just like plain trySelect, but support idempotent start - // Returns RESUME_TOKEN | RETRY_ATOMIC | null (when already selected) - override fun trySelectOther(otherOp: PrepareOp?): Any? { - _state.loop { state -> // lock-free loop on state - when { - // Found initial state (not selected yet) -- try to make it selected - state === NOT_SELECTED -> { - if (otherOp == null) { - // regular trySelect -- just mark as select - if (!_state.compareAndSet(NOT_SELECTED, null)) return@loop - } else { - // Rendezvous with another select instance -- install PairSelectOp - val pairSelectOp = PairSelectOp(otherOp) - if (!_state.compareAndSet(NOT_SELECTED, pairSelectOp)) return@loop - val decision = pairSelectOp.perform(this) - if (decision !== null) return decision + fun trySelectDetailed(clauseObject: Any, result: Any?) = + TrySelectDetailedResult(trySelectInternal(clauseObject, result)) + + private fun trySelectInternal(clauseObject: Any, internalResult: Any?): Int { + while (true) { + when (val curState = state.value) { + // Perform a rendezvous with this select if it is in WAITING state. + is CancellableContinuation<*> -> { + val clause = findClause(clauseObject) ?: continue // retry if `clauses` is already `null` + val onCancellation = clause.createOnCancellationAction(this@SelectImplementation, internalResult) + if (state.compareAndSet(curState, clause)) { + @Suppress("UNCHECKED_CAST") + val cont = curState as CancellableContinuation<Unit> + // Success! Store the resumption value and + // try to resume the continuation. + this.internalResult = internalResult + if (cont.tryResume(onCancellation)) return TRY_SELECT_SUCCESSFUL + // If the resumption failed, we need to clean + // the [result] field to avoid memory leaks. + this.internalResult = null + return TRY_SELECT_CANCELLED } - doAfterSelect() - return RESUME_TOKEN } - state is OpDescriptor -> { // state is either AtomicSelectOp or PairSelectOp - // Found descriptor of ongoing operation while working in the context of other select operation - if (otherOp != null) { - val otherAtomicOp = otherOp.atomicOp - when { - // It is the same select instance - otherAtomicOp is AtomicSelectOp && otherAtomicOp.impl === this -> { - /* - * We cannot do state.perform(this) here and "help" it since it is the same - * select and we'll get StackOverflowError. - * See https://github.com/Kotlin/kotlinx.coroutines/issues/1411 - * We cannot support this because select { ... } is an expression and its clauses - * have a result that shall be returned from the select. - */ - error("Cannot use matching select clauses on the same object") - } - // The other select (that is trying to proceed) had started earlier - otherAtomicOp.isEarlierThan(state) -> { - /** - * Abort to prevent deadlock by returning a failure to it. - * See https://github.com/Kotlin/kotlinx.coroutines/issues/504 - * The other select operation will receive a failure and will restart itself with a - * larger sequence number. This guarantees obstruction-freedom of this algorithm. - */ - return RETRY_ATOMIC - } - } - } - // Otherwise (not a special descriptor) - state.perform(this) // help it - } - // otherwise -- already selected - otherOp == null -> return null // already selected - state === otherOp.desc -> return RESUME_TOKEN // was selected with this marker - else -> return null // selected with different marker + // Already selected. + STATE_COMPLETED, is SelectImplementation<*>.ClauseData -> return TRY_SELECT_ALREADY_SELECTED + // Already cancelled. + STATE_CANCELLED -> return TRY_SELECT_CANCELLED + // This select is still in REGISTRATION phase, re-register the clause + // in order not to wait until this select moves to WAITING phase. + // This is a rare race, so we do not need to worry about performance here. + STATE_REG -> if (state.compareAndSet(curState, listOf(clauseObject))) return TRY_SELECT_REREGISTER + // This select is still in REGISTRATION phase, and the state stores a list of clauses + // for re-registration, add the selecting clause to this list. + // This is a rare race, so we do not need to worry about performance here. + is List<*> -> if (state.compareAndSet(curState, curState + clauseObject)) return TRY_SELECT_REREGISTER + // Another state? Something went really wrong. + else -> error("Unexpected state: $curState") } } } - // The very last step of rendezvous between two select operations - private class PairSelectOp( - @JvmField val otherOp: PrepareOp - ) : OpDescriptor() { - override fun perform(affected: Any?): Any? { - val impl = affected as SelectBuilderImpl<*> - // here we are definitely not going to RETRY_ATOMIC, so - // we must finish preparation of another operation before attempting to reach decision to select - otherOp.finishPrepare() - val decision = otherOp.atomicOp.decide(null) // try decide for success of operation - val update: Any = if (decision == null) otherOp.desc else NOT_SELECTED - impl._state.compareAndSet(this, update) - return decision - } - - override val atomicOp: AtomicOp<*> - get() = otherOp.atomicOp + /** + * Finds the clause with the corresponding [clause object][SelectClause.clauseObject]. + * If the reference to the list of clauses is already cleared due to completion/cancellation, + * this function returns `null` + */ + private fun findClause(clauseObject: Any): ClauseData? { + // Read the list of clauses. If the `clauses` field is already `null`, + // the clean-up phase has already completed, and this function returns `null`. + val clauses = this.clauses ?: return null + // Find the clause with the specified clause object. + return clauses.find { it.clauseObject === clauseObject } + ?: error("Clause with object $clauseObject is not found") } - override fun performAtomicTrySelect(desc: AtomicDesc): Any? = - AtomicSelectOp(this, desc).perform(null) - - override fun toString(): String = "SelectInstance(state=${_state.value}, result=${_result.value})" - - private class AtomicSelectOp( - @JvmField val impl: SelectBuilderImpl<*>, - @JvmField val desc: AtomicDesc - ) : AtomicOp<Any?>() { - // all select operations are totally ordered by their creating time using selectOpSequenceNumber - override val opSequence = selectOpSequenceNumber.next() + // ============== + // = COMPLETION = + // ============== - init { - desc.atomicOp = this + /** + * Completes this `select` operation after the internal result is provided + * via [SelectInstance.trySelect] or [SelectInstance.selectInRegistrationPhase]. + * (1) First, this function applies the [ProcessResultFunction] of the selected clause + * to the internal result. + * (2) After that, the [clean-up procedure][cleanup] + * is called to remove this `select` instance from other clause objects, and + * make it possible to collect it by GC after this `select` finishes. + * (3) Finally, the user-specified block is invoked + * with the processed result as an argument. + */ + private suspend fun complete(): R { + assert { isSelected } + // Get the selected clause. + @Suppress("UNCHECKED_CAST") + val selectedClause = state.value as SelectImplementation<R>.ClauseData + // Perform the clean-up before the internal result processing and + // the user-specified block invocation to guarantee the absence + // of memory leaks. Collect the internal result before that. + val internalResult = this.internalResult + cleanup(selectedClause) + // Process the internal result and invoke the user's block. + return if (!RECOVER_STACK_TRACES) { + // TAIL-CALL OPTIMIZATION: the `suspend` block + // is invoked at the very end. + val blockArgument = selectedClause.processResult(internalResult) + selectedClause.invokeBlock(blockArgument) + } else { + // TAIL-CALL OPTIMIZATION: the `suspend` + // function is invoked at the very end. + // However, internally this `suspend` function + // constructs a state machine to recover a + // possible stack-trace. + processResultAndInvokeBlockRecoveringException(selectedClause, internalResult) } + } - override fun prepare(affected: Any?): Any? { - // only originator of operation makes preparation move of installing descriptor into this selector's state - // helpers should never do it, or risk ruining progress when they come late - if (affected == null) { - // we are originator (affected reference is not null if helping) - prepareSelectOp()?.let { return it } - } - try { - return desc.prepare(this) - } catch (e: Throwable) { - // undo prepareSelectedOp on crash (for example if IllegalStateException is thrown) - if (affected == null) undoPrepare() - throw e - } + private suspend fun processResultAndInvokeBlockRecoveringException(clause: ClauseData, internalResult: Any?): R = + try { + val blockArgument = clause.processResult(internalResult) + clause.invokeBlock(blockArgument) + } catch (e: Throwable) { + // In the debug mode, we need to properly recover + // the stack-trace of the exception; the tail-call + // optimization cannot be applied here. + recoverAndThrow(e) } - override fun complete(affected: Any?, failure: Any?) { - completeSelect(failure) - desc.complete(this, failure) + /** + * Invokes all [DisposableHandle]-s provided via + * [SelectInstance.disposeOnCompletion] during + * clause registrations. + */ + private fun cleanup(selectedClause: ClauseData) { + assert { state.value == selectedClause } + // Read the list of clauses. If the `clauses` field is already `null`, + // a concurrent clean-up procedure has already completed, and it is safe to finish. + val clauses = this.clauses ?: return + // Invoke all cancellation handlers except for the + // one related to the selected clause, if specified. + clauses.forEach { clause -> + if (clause !== selectedClause) clause.dispose() } + // We do need to clean all the data to avoid memory leaks. + this.state.value = STATE_COMPLETED + this.internalResult = NO_RESULT + this.clauses = null + } - private fun prepareSelectOp(): Any? { - impl._state.loop { state -> - when { - state === this -> return null // already in progress - state is OpDescriptor -> state.perform(impl) // help - state === NOT_SELECTED -> { - if (impl._state.compareAndSet(NOT_SELECTED, this)) - return null // success - } - else -> return ALREADY_SELECTED - } - } + // [CompletionHandler] implementation, must be invoked on cancellation. + override fun invoke(cause: Throwable?) { + // Update the state. + state.update { cur -> + // Finish immediately when this `select` is already completed. + // Notably, this select might be logically completed + // (the `state` field stores the selected `ClauseData`), + // while the continuation is already cancelled. + // We need to invoke the cancellation handler in this case. + if (cur === STATE_COMPLETED) return + STATE_CANCELLED } + // Read the list of clauses. If the `clauses` field is already `null`, + // a concurrent clean-up procedure has already completed, and it is safe to finish. + val clauses = this.clauses ?: return + // Remove this `select` instance from all the clause object (channels, mutexes, etc.). + clauses.forEach { it.dispose() } + // We do need to clean all the data to avoid memory leaks. + this.internalResult = NO_RESULT + this.clauses = null + } - // reverts the change done by prepareSelectedOp - private fun undoPrepare() { - impl._state.compareAndSet(this, NOT_SELECTED) + /** + * Each `select` clause is internally represented with a [ClauseData] instance. + */ + internal inner class ClauseData( + @JvmField val clauseObject: Any, // the object of this `select` clause: Channel, Mutex, Job, ... + private val regFunc: RegistrationFunction, + private val processResFunc: ProcessResultFunction, + private val param: Any?, // the user-specified param + private val block: Any, // the user-specified block, which should be called if this clause becomes selected + @JvmField val onCancellationConstructor: OnCancellationConstructor? + ) { + @JvmField var disposableHandleOrSegment: Any? = null + @JvmField var indexInSegment: Int = -1 + + /** + * Tries to register the specified [select] instance in [clauseObject] and check + * whether the registration succeeded or a rendezvous has happened during the registration. + * This function returns `true` if this [select] is successfully registered and + * is _waiting_ for a rendezvous, or `false` when this clause becomes + * selected during registration. + * + * For example, the [Channel.onReceive] clause registration + * on a non-empty channel retrieves the first element and completes + * the corresponding [select] via [SelectInstance.selectInRegistrationPhase]. + */ + fun tryRegisterAsWaiter(select: SelectImplementation<R>): Boolean { + assert { select.inRegistrationPhase || select.isCancelled } + assert { select.internalResult === NO_RESULT } + regFunc(clauseObject, select, param) + return select.internalResult === NO_RESULT } - private fun completeSelect(failure: Any?) { - val selectSuccess = failure == null - val update = if (selectSuccess) null else NOT_SELECTED - if (impl._state.compareAndSet(this, update)) { - if (selectSuccess) - impl.doAfterSelect() + /** + * Processes the internal result provided via either + * [SelectInstance.selectInRegistrationPhase] or + * [SelectInstance.trySelect] and returns an argument + * for the user-specified [block]. + * + * Importantly, this function may throw an exception + * (e.g., when the channel is closed in [Channel.onSend], the + * corresponding [ProcessResultFunction] is bound to fail). + */ + fun processResult(result: Any?) = processResFunc(clauseObject, param, result) + + /** + * Invokes the user-specified block and returns + * the final result of this `select` clause. + */ + @Suppress("UNCHECKED_CAST") + suspend fun invokeBlock(argument: Any?): R { + val block = block + // We distinguish no-argument and 1-argument + // lambdas via special markers for the clause + // parameters. Specifically, PARAM_CLAUSE_0 + // is always used with [SelectClause0], which + // takes a no-argument lambda. + // + // TAIL-CALL OPTIMIZATION: we invoke + // the `suspend` block at the very end. + return if (this.param === PARAM_CLAUSE_0) { + block as suspend () -> R + block() + } else { + block as suspend (Any?) -> R + block(argument) } } - override fun toString(): String = "AtomicSelectOp(sequence=$opSequence)" - } - - override fun SelectClause0.invoke(block: suspend () -> R) { - registerSelectClause0(this@SelectBuilderImpl, block) - } - - override fun <Q> SelectClause1<Q>.invoke(block: suspend (Q) -> R) { - registerSelectClause1(this@SelectBuilderImpl, block) - } + fun dispose() { + with(disposableHandleOrSegment) { + if (this is Segment<*>) { + this.onCancellation(indexInSegment, null, context) + } else { + (this as? DisposableHandle)?.dispose() + } + } + } - override fun <P, Q> SelectClause2<P, Q>.invoke(param: P, block: suspend (Q) -> R) { - registerSelectClause2(this@SelectBuilderImpl, param, block) + fun createOnCancellationAction(select: SelectInstance<*>, internalResult: Any?) = + onCancellationConstructor?.invoke(select, param, internalResult) } +} - override fun onTimeout(timeMillis: Long, block: suspend () -> R) { - if (timeMillis <= 0L) { - if (trySelect()) - block.startCoroutineUnintercepted(completion) - return - } - val action = Runnable { - // todo: we could have replaced startCoroutine with startCoroutineUndispatched - // But we need a way to know that Delay.invokeOnTimeout had used the right thread - if (trySelect()) - block.startCoroutineCancellable(completion) // shall be cancellable while waits for dispatch - } - disposeOnSelect(context.delay.invokeOnTimeout(timeMillis, action, context)) - } +private fun CancellableContinuation<Unit>.tryResume(onCancellation: ((cause: Throwable) -> Unit)?): Boolean { + val token = tryResume(Unit, null, onCancellation) ?: return false + completeResume(token) + return true +} - private class DisposeNode( - @JvmField val handle: DisposableHandle - ) : LockFreeLinkedListNode() +// trySelectInternal(..) results. +private const val TRY_SELECT_SUCCESSFUL = 0 +private const val TRY_SELECT_REREGISTER = 1 +private const val TRY_SELECT_CANCELLED = 2 +private const val TRY_SELECT_ALREADY_SELECTED = 3 +// trySelectDetailed(..) results. +internal enum class TrySelectDetailedResult { + SUCCESSFUL, REREGISTER, CANCELLED, ALREADY_SELECTED +} +private fun TrySelectDetailedResult(trySelectInternalResult: Int): TrySelectDetailedResult = when(trySelectInternalResult) { + TRY_SELECT_SUCCESSFUL -> SUCCESSFUL + TRY_SELECT_REREGISTER -> REREGISTER + TRY_SELECT_CANCELLED -> CANCELLED + TRY_SELECT_ALREADY_SELECTED -> ALREADY_SELECTED + else -> error("Unexpected internal result: $trySelectInternalResult") } + +// Markers for REGISTRATION, COMPLETED, and CANCELLED states. +private val STATE_REG = Symbol("STATE_REG") +private val STATE_COMPLETED = Symbol("STATE_COMPLETED") +private val STATE_CANCELLED = Symbol("STATE_CANCELLED") +// As the selection result is nullable, we use this special +// marker for the absence of result. +private val NO_RESULT = Symbol("NO_RESULT") +// We use this marker parameter objects to distinguish +// SelectClause[0,1,2] and invoke the user-specified block correctly. +internal val PARAM_CLAUSE_0 = Symbol("PARAM_CLAUSE_0") diff --git a/kotlinx-coroutines-core/common/src/selects/SelectOld.kt b/kotlinx-coroutines-core/common/src/selects/SelectOld.kt new file mode 100644 index 00000000..85476d2c --- /dev/null +++ b/kotlinx-coroutines-core/common/src/selects/SelectOld.kt @@ -0,0 +1,147 @@ +/* + * Copyright 2016-2021 JetBrains s.r.o. Use of this source code is governed by the Apache 2.0 license. + */ + +package kotlinx.coroutines.selects + +import kotlinx.coroutines.* +import kotlin.coroutines.* +import kotlin.coroutines.intrinsics.* + +/* + * For binary compatibility, we need to maintain the previous `select` implementations. + * Thus, we keep [SelectBuilderImpl] and [UnbiasedSelectBuilderImpl] and implement the + * functions marked with `@PublishedApi`. + * + * We keep the old `select` functions as [selectOld] and [selectUnbiasedOld] for test purpose. + */ + +@PublishedApi +internal class SelectBuilderImpl<R>( + uCont: Continuation<R> // unintercepted delegate continuation +) : SelectImplementation<R>(uCont.context) { + private val cont = CancellableContinuationImpl(uCont.intercepted(), MODE_CANCELLABLE) + + @PublishedApi + internal fun getResult(): Any? { + // In the current `select` design, the [select] and [selectUnbiased] functions + // do not wrap the operation in `suspendCoroutineUninterceptedOrReturn` and + // suspend explicitly via [doSelect] call, which returns the final result. + // However, [doSelect] is a suspend function, so it cannot be invoked directly. + // In addition, the `select` builder is eligible to throw an exception, which + // should be handled properly. + // + // As a solution, we: + // 1) check whether the `select` building is already completed with exception, finishing immediately in this case; + // 2) create a CancellableContinuationImpl with the provided unintercepted continuation as a delegate; + // 3) wrap the [doSelect] call in an additional coroutine, which we launch in UNDISPATCHED mode; + // 4) resume the created CancellableContinuationImpl after the [doSelect] invocation completes; + // 5) use CancellableContinuationImpl.getResult() as a result of this function. + if (cont.isCompleted) return cont.getResult() + CoroutineScope(context).launch(start = CoroutineStart.UNDISPATCHED) { + val result = try { + doSelect() + } catch (e: Throwable) { + cont.resumeUndispatchedWithException(e) + return@launch + } + cont.resumeUndispatched(result) + } + return cont.getResult() + } + + @PublishedApi + internal fun handleBuilderException(e: Throwable) { + cont.resumeWithException(e) // will be thrown later via `cont.getResult()` + } +} + +@PublishedApi +internal class UnbiasedSelectBuilderImpl<R>( + uCont: Continuation<R> // unintercepted delegate continuation +) : UnbiasedSelectImplementation<R>(uCont.context) { + private val cont = CancellableContinuationImpl(uCont.intercepted(), MODE_CANCELLABLE) + + @PublishedApi + internal fun initSelectResult(): Any? { + // Here, we do the same trick as in [SelectBuilderImpl]. + if (cont.isCompleted) return cont.getResult() + CoroutineScope(context).launch(start = CoroutineStart.UNDISPATCHED) { + val result = try { + doSelect() + } catch (e: Throwable) { + cont.resumeUndispatchedWithException(e) + return@launch + } + cont.resumeUndispatched(result) + } + return cont.getResult() + } + + @PublishedApi + internal fun handleBuilderException(e: Throwable) { + cont.resumeWithException(e) + } +} + +/* + * This is the old version of `select`. It should work to guarantee binary compatibility. + * + * Internal note: + * We do test it manually by changing the implementation of **new** select with the following: + * ``` + * public suspend inline fun <R> select(crossinline builder: SelectBuilder<R>.() -> Unit): R { + * contract { + * callsInPlace(builder, InvocationKind.EXACTLY_ONCE) + * } + * return selectOld(builder) + * } + * ``` + * + * These signatures are not used by the already compiled code, but their body is. + */ +@PublishedApi +internal suspend inline fun <R> selectOld(crossinline builder: SelectBuilder<R>.() -> Unit): R { + return suspendCoroutineUninterceptedOrReturn { uCont -> + val scope = SelectBuilderImpl(uCont) + try { + builder(scope) + } catch (e: Throwable) { + scope.handleBuilderException(e) + } + scope.getResult() + } +} + +// This is the old version of `selectUnbiased`. It should work to guarantee binary compatibility. +@PublishedApi +internal suspend inline fun <R> selectUnbiasedOld(crossinline builder: SelectBuilder<R>.() -> Unit): R = + suspendCoroutineUninterceptedOrReturn { uCont -> + val scope = UnbiasedSelectBuilderImpl(uCont) + try { + builder(scope) + } catch (e: Throwable) { + scope.handleBuilderException(e) + } + scope.initSelectResult() + } + +@OptIn(ExperimentalStdlibApi::class) +private fun <T> CancellableContinuation<T>.resumeUndispatched(result: T) { + val dispatcher = context[CoroutineDispatcher] + if (dispatcher != null) { + dispatcher.resumeUndispatched(result) + } else { + resume(result) + } +} + +@OptIn(ExperimentalStdlibApi::class) +private fun CancellableContinuation<*>.resumeUndispatchedWithException(exception: Throwable) { + val dispatcher = context[CoroutineDispatcher] + if (dispatcher != null) { + dispatcher.resumeUndispatchedWithException(exception) + } else { + resumeWithException(exception) + } +} diff --git a/kotlinx-coroutines-core/common/src/selects/SelectUnbiased.kt b/kotlinx-coroutines-core/common/src/selects/SelectUnbiased.kt index c33c5b1f..a4b1e043 100644 --- a/kotlinx-coroutines-core/common/src/selects/SelectUnbiased.kt +++ b/kotlinx-coroutines-core/common/src/selects/SelectUnbiased.kt @@ -1,11 +1,12 @@ /* * Copyright 2016-2021 JetBrains s.r.o. Use of this source code is governed by the Apache 2.0 license. */ +@file:OptIn(ExperimentalContracts::class) package kotlinx.coroutines.selects +import kotlin.contracts.* import kotlin.coroutines.* -import kotlin.coroutines.intrinsics.* /** * Waits for the result of multiple suspending functions simultaneously like [select], but in an _unbiased_ @@ -17,53 +18,50 @@ import kotlin.coroutines.intrinsics.* * * See [select] function description for all the other details. */ -public suspend inline fun <R> selectUnbiased(crossinline builder: SelectBuilder<R>.() -> Unit): R = - suspendCoroutineUninterceptedOrReturn { uCont -> - val scope = UnbiasedSelectBuilderImpl(uCont) - try { - builder(scope) - } catch (e: Throwable) { - scope.handleBuilderException(e) - } - scope.initSelectResult() +@OptIn(ExperimentalContracts::class) +public suspend inline fun <R> selectUnbiased(crossinline builder: SelectBuilder<R>.() -> Unit): R { + contract { + callsInPlace(builder, InvocationKind.EXACTLY_ONCE) } + return UnbiasedSelectImplementation<R>(coroutineContext).run { + builder(this) + doSelect() + } +} - +/** + * The unbiased `select` inherits the [standard one][SelectImplementation], + * but does not register clauses immediately. Instead, it stores all of them + * in [clausesToRegister] lists, shuffles and registers them in the beginning of [doSelect] + * (see [shuffleAndRegisterClauses]), and then delegates the rest + * to the parent's [doSelect] implementation. + */ @PublishedApi -internal class UnbiasedSelectBuilderImpl<in R>(uCont: Continuation<R>) : - SelectBuilder<R> { - val instance = SelectBuilderImpl(uCont) - val clauses = arrayListOf<() -> Unit>() - - @PublishedApi - internal fun handleBuilderException(e: Throwable): Unit = instance.handleBuilderException(e) - - @PublishedApi - internal fun initSelectResult(): Any? { - if (!instance.isSelected) { - try { - clauses.shuffle() - clauses.forEach { it.invoke() } - } catch (e: Throwable) { - instance.handleBuilderException(e) - } - } - return instance.getResult() - } +internal open class UnbiasedSelectImplementation<R>(context: CoroutineContext) : SelectImplementation<R>(context) { + private val clausesToRegister: MutableList<ClauseData> = arrayListOf() override fun SelectClause0.invoke(block: suspend () -> R) { - clauses += { registerSelectClause0(instance, block) } + clausesToRegister += ClauseData(clauseObject, regFunc, processResFunc, PARAM_CLAUSE_0, block, onCancellationConstructor) } override fun <Q> SelectClause1<Q>.invoke(block: suspend (Q) -> R) { - clauses += { registerSelectClause1(instance, block) } + clausesToRegister += ClauseData(clauseObject, regFunc, processResFunc, null, block, onCancellationConstructor) } override fun <P, Q> SelectClause2<P, Q>.invoke(param: P, block: suspend (Q) -> R) { - clauses += { registerSelectClause2(instance, param, block) } + clausesToRegister += ClauseData(clauseObject, regFunc, processResFunc, param, block, onCancellationConstructor) + } + + @PublishedApi + override suspend fun doSelect(): R { + shuffleAndRegisterClauses() + return super.doSelect() } - override fun onTimeout(timeMillis: Long, block: suspend () -> R) { - clauses += { instance.onTimeout(timeMillis, block) } + private fun shuffleAndRegisterClauses() = try { + clausesToRegister.shuffle() + clausesToRegister.forEach { it.register() } + } finally { + clausesToRegister.clear() } } diff --git a/kotlinx-coroutines-core/common/src/selects/WhileSelect.kt b/kotlinx-coroutines-core/common/src/selects/WhileSelect.kt index 98a9c672..ccda6568 100644 --- a/kotlinx-coroutines-core/common/src/selects/WhileSelect.kt +++ b/kotlinx-coroutines-core/common/src/selects/WhileSelect.kt @@ -28,5 +28,5 @@ import kotlinx.coroutines.* */ @ExperimentalCoroutinesApi public suspend inline fun whileSelect(crossinline builder: SelectBuilder<Boolean>.() -> Unit) { - while(select<Boolean>(builder)) {} + while(select(builder)) { /* do nothing */ } } diff --git a/kotlinx-coroutines-core/common/src/sync/Mutex.kt b/kotlinx-coroutines-core/common/src/sync/Mutex.kt index 681d5db6..06aebcbe 100644 --- a/kotlinx-coroutines-core/common/src/sync/Mutex.kt +++ b/kotlinx-coroutines-core/common/src/sync/Mutex.kt @@ -7,11 +7,9 @@ package kotlinx.coroutines.sync import kotlinx.atomicfu.* import kotlinx.coroutines.* import kotlinx.coroutines.internal.* -import kotlinx.coroutines.intrinsics.* import kotlinx.coroutines.selects.* import kotlin.contracts.* import kotlin.jvm.* -import kotlin.native.concurrent.* /** * Mutual exclusion for coroutines. @@ -22,18 +20,22 @@ import kotlin.native.concurrent.* * * JVM API note: * Memory semantic of the [Mutex] is similar to `synchronized` block on JVM: - * An unlock on a [Mutex] happens-before every subsequent successful lock on that [Mutex]. + * An unlock operation on a [Mutex] happens-before every subsequent successful lock on that [Mutex]. * Unsuccessful call to [tryLock] do not have any memory effects. */ public interface Mutex { /** - * Returns `true` when this mutex is locked. + * Returns `true` if this mutex is locked. */ public val isLocked: Boolean /** * Tries to lock this mutex, returning `false` if this mutex is already locked. * + * It is recommended to use [withLock] for safety reasons, so that the acquired lock is always + * released at the end of your critical section, and [unlock] is never invoked before a successful + * lock acquisition. + * * @param owner Optional owner token for debugging. When `owner` is specified (non-null value) and this mutex * is already locked with the same token (same identity), this function throws [IllegalStateException]. */ @@ -52,26 +54,33 @@ public interface Mutex { * Note that this function does not check for cancellation when it is not suspended. * Use [yield] or [CoroutineScope.isActive] to periodically check for cancellation in tight loops if needed. * - * Use [tryLock] to try acquiring a lock without waiting. + * Use [tryLock] to try acquiring the lock without waiting. * * This function is fair; suspended callers are resumed in first-in-first-out order. * + * It is recommended to use [withLock] for safety reasons, so that the acquired lock is always + * released at the end of the critical section, and [unlock] is never invoked before a successful + * lock acquisition. + * * @param owner Optional owner token for debugging. When `owner` is specified (non-null value) and this mutex * is already locked with the same token (same identity), this function throws [IllegalStateException]. */ public suspend fun lock(owner: Any? = null) /** - * Deprecated for removal without built-in replacement. + * Clause for [select] expression of [lock] suspending function that selects when the mutex is locked. + * Additional parameter for the clause in the `owner` (see [lock]) and when the clause is selected + * the reference to this mutex is passed into the corresponding block. */ @Deprecated(level = DeprecationLevel.WARNING, message = "Mutex.onLock deprecated without replacement. " + "For additional details please refer to #2794") // WARNING since 1.6.0 public val onLock: SelectClause2<Any?, Mutex> /** - * Checks mutex locked by owner + * Checks whether this mutex is locked by the specified owner. * - * @return `true` on mutex lock by owner, `false` if not locker or it is locked by different owner + * @return `true` when this mutex is locked by the specified owner; + * `false` if the mutex is not locked or locked by another owner. */ public fun holdsLock(owner: Any): Boolean @@ -79,6 +88,10 @@ public interface Mutex { * Unlocks this mutex. Throws [IllegalStateException] if invoked on a mutex that is not locked or * was locked with a different owner token (by identity). * + * It is recommended to use [withLock] for safety reasons, so that the acquired lock is always + * released at the end of the critical section, and [unlock] is never invoked before a successful + * lock acquisition. + * * @param owner Optional owner token for debugging. When `owner` is specified (non-null value) and this mutex * was locked with the different token (by identity), this function throws [IllegalStateException]. */ @@ -105,7 +118,7 @@ public fun Mutex(locked: Boolean = false): Mutex = */ @OptIn(ExperimentalContracts::class) public suspend inline fun <T> Mutex.withLock(owner: Any? = null, action: () -> T): T { - contract { + contract { callsInPlace(action, InvocationKind.EXACTLY_ONCE) } @@ -117,307 +130,180 @@ public suspend inline fun <T> Mutex.withLock(owner: Any? = null, action: () -> T } } -@SharedImmutable -private val LOCK_FAIL = Symbol("LOCK_FAIL") -@SharedImmutable -private val UNLOCK_FAIL = Symbol("UNLOCK_FAIL") -@SharedImmutable -private val LOCKED = Symbol("LOCKED") -@SharedImmutable -private val UNLOCKED = Symbol("UNLOCKED") - -@SharedImmutable -private val EMPTY_LOCKED = Empty(LOCKED) -@SharedImmutable -private val EMPTY_UNLOCKED = Empty(UNLOCKED) - -private class Empty( - @JvmField val locked: Any -) { - override fun toString(): String = "Empty[$locked]" -} -internal class MutexImpl(locked: Boolean) : Mutex, SelectClause2<Any?, Mutex> { - // State is: Empty | LockedQueue | OpDescriptor - // shared objects while we have no waiters - private val _state = atomic<Any?>(if (locked) EMPTY_LOCKED else EMPTY_UNLOCKED) - - public override val isLocked: Boolean get() { - _state.loop { state -> - when (state) { - is Empty -> return state.locked !== UNLOCKED - is LockedQueue -> return true - is OpDescriptor -> state.perform(this) // help - else -> error("Illegal state $state") - } +internal open class MutexImpl(locked: Boolean) : SemaphoreImpl(1, if (locked) 1 else 0), Mutex { + /** + * After the lock is acquired, the corresponding owner is stored in this field. + * The [unlock] operation checks the owner and either re-sets it to [NO_OWNER], + * if there is no waiting request, or to the owner of the suspended [lock] operation + * to be resumed, otherwise. + */ + private val owner = atomic<Any?>(if (locked) null else NO_OWNER) + + private val onSelectCancellationUnlockConstructor: OnCancellationConstructor = + { _: SelectInstance<*>, owner: Any?, _: Any? -> + { unlock(owner) } } - } - // for tests ONLY - internal val isLockedEmptyQueueState: Boolean get() { - val state = _state.value - return state is LockedQueue && state.isEmpty - } + override val isLocked: Boolean get() = + availablePermits == 0 - public override fun tryLock(owner: Any?): Boolean { - _state.loop { state -> - when (state) { - is Empty -> { - if (state.locked !== UNLOCKED) return false - val update = if (owner == null) EMPTY_LOCKED else Empty( - owner - ) - if (_state.compareAndSet(state, update)) return true - } - is LockedQueue -> { - check(state.owner !== owner) { "Already locked by $owner" } - return false - } - is OpDescriptor -> state.perform(this) // help - else -> error("Illegal state $state") - } + override fun holdsLock(owner: Any): Boolean = holdsLockImpl(owner) == HOLDS_LOCK_YES + + /** + * [HOLDS_LOCK_UNLOCKED] if the mutex is unlocked + * [HOLDS_LOCK_YES] if the mutex is held with the specified [owner] + * [HOLDS_LOCK_ANOTHER_OWNER] if the mutex is held with a different owner + */ + private fun holdsLockImpl(owner: Any?): Int { + while (true) { + // Is this mutex locked? + if (!isLocked) return HOLDS_LOCK_UNLOCKED + val curOwner = this.owner.value + // Wait in a spin-loop until the owner is set + if (curOwner === NO_OWNER) continue // <-- ATTENTION, BLOCKING PART HERE + // Check the owner + return if (curOwner === owner) HOLDS_LOCK_YES else HOLDS_LOCK_ANOTHER_OWNER } } - public override suspend fun lock(owner: Any?) { - // fast-path -- try lock + override suspend fun lock(owner: Any?) { if (tryLock(owner)) return - // slow-path -- suspend - return lockSuspend(owner) + lockSuspend(owner) } - private suspend fun lockSuspend(owner: Any?) = suspendCancellableCoroutineReusable<Unit> sc@ { cont -> - var waiter = LockCont(owner, cont) - _state.loop { state -> - when (state) { - is Empty -> { - if (state.locked !== UNLOCKED) { // try upgrade to queue & retry - _state.compareAndSet(state, LockedQueue(state.locked)) - } else { - // try lock - val update = if (owner == null) EMPTY_LOCKED else Empty(owner) - if (_state.compareAndSet(state, update)) { // locked - // TODO implement functional type in LockCont as soon as we get rid of legacy JS - cont.resume(Unit) { unlock(owner) } - return@sc - } - } - } - is LockedQueue -> { - val curOwner = state.owner - check(curOwner !== owner) { "Already locked by $owner" } - - state.addLast(waiter) - /* - * If the state has been changed while we were adding the waiter, - * it means that 'unlock' has taken it and _either_ resumed it successfully or just overwritten. - * To rendezvous that, we try to "invalidate" our node and go for retry. - * - * Node has to be re-instantiated as we do not support node re-adding, even to - * another list - */ - if (_state.value === state || !waiter.take()) { - // added to waiter list - cont.removeOnCancellation(waiter) - return@sc - } - - waiter = LockCont(owner, cont) - return@loop - } - is OpDescriptor -> state.perform(this) // help - else -> error("Illegal state $state") - } - } + private suspend fun lockSuspend(owner: Any?) = suspendCancellableCoroutineReusable<Unit> { cont -> + val contWithOwner = CancellableContinuationWithOwner(cont, owner) + acquire(contWithOwner) } - override val onLock: SelectClause2<Any?, Mutex> - get() = this - - // registerSelectLock - @Suppress("PARAMETER_NAME_CHANGED_ON_OVERRIDE") - override fun <R> registerSelectClause2(select: SelectInstance<R>, owner: Any?, block: suspend (Mutex) -> R) { - while (true) { // lock-free loop on state - if (select.isSelected) return - when (val state = _state.value) { - is Empty -> { - if (state.locked !== UNLOCKED) { // try upgrade to queue & retry - _state.compareAndSet(state, LockedQueue(state.locked)) - } else { - // try lock - val failure = select.performAtomicTrySelect(TryLockDesc(this, owner)) - when { - failure == null -> { // success - block.startCoroutineUnintercepted(receiver = this, completion = select.completion) - return - } - failure === ALREADY_SELECTED -> return // already selected -- bail out - failure === LOCK_FAIL -> {} // retry - failure === RETRY_ATOMIC -> {} // retry - else -> error("performAtomicTrySelect(TryLockDesc) returned $failure") - } - } - } - is LockedQueue -> { - check(state.owner !== owner) { "Already locked by $owner" } - val node = LockSelect(owner, select, block) - /* - * If the state has been changed while we were adding the waiter, - * it means that 'unlock' has taken it and _either_ resumed it successfully or just overwritten. - * To rendezvous that, we try to "invalidate" our node and go for retry. - * - * Node has to be re-instantiated as we do not support node re-adding, even to - * another list - */ - state.addLast(node) - if (_state.value === state || !node.take()) { - // added to waiter list - select.disposeOnSelect(node) - return - } - } - is OpDescriptor -> state.perform(this) // help - else -> error("Illegal state $state") - } - } + override fun tryLock(owner: Any?): Boolean = when (tryLockImpl(owner)) { + TRY_LOCK_SUCCESS -> true + TRY_LOCK_FAILED -> false + TRY_LOCK_ALREADY_LOCKED_BY_OWNER -> error("This mutex is already locked by the specified owner: $owner") + else -> error("unexpected") } - private class TryLockDesc( - @JvmField val mutex: MutexImpl, - @JvmField val owner: Any? - ) : AtomicDesc() { - // This is Harris's RDCSS (Restricted Double-Compare Single Swap) operation - private inner class PrepareOp(override val atomicOp: AtomicOp<*>) : OpDescriptor() { - override fun perform(affected: Any?): Any? { - val update: Any = if (atomicOp.isDecided) EMPTY_UNLOCKED else atomicOp // restore if was already decided - (affected as MutexImpl)._state.compareAndSet(this, update) - return null // ok - } - } - - override fun prepare(op: AtomicOp<*>): Any? { - val prepare = PrepareOp(op) - if (!mutex._state.compareAndSet(EMPTY_UNLOCKED, prepare)) return LOCK_FAIL - return prepare.perform(mutex) - } - - override fun complete(op: AtomicOp<*>, failure: Any?) { - val update = if (failure != null) EMPTY_UNLOCKED else { - if (owner == null) EMPTY_LOCKED else Empty(owner) + private fun tryLockImpl(owner: Any?): Int { + while (true) { + if (tryAcquire()) { + assert { this.owner.value === NO_OWNER } + this.owner.value = owner + return TRY_LOCK_SUCCESS + } else { + // The semaphore permit acquisition has failed. + // However, we need to check that this mutex is not + // locked by our owner. + if (owner == null) return TRY_LOCK_FAILED + when (holdsLockImpl(owner)) { + // This mutex is already locked by our owner. + HOLDS_LOCK_YES -> return TRY_LOCK_ALREADY_LOCKED_BY_OWNER + // This mutex is locked by another owner, `trylock(..)` must return `false`. + HOLDS_LOCK_ANOTHER_OWNER -> return TRY_LOCK_FAILED + // This mutex is no longer locked, restart the operation. + HOLDS_LOCK_UNLOCKED -> continue + } } - mutex._state.compareAndSet(op, update) } } - public override fun holdsLock(owner: Any) = - _state.value.let { state -> - when (state) { - is Empty -> state.locked === owner - is LockedQueue -> state.owner === owner - else -> false - } - } - override fun unlock(owner: Any?) { - _state.loop { state -> - when (state) { - is Empty -> { - if (owner == null) - check(state.locked !== UNLOCKED) { "Mutex is not locked" } - else - check(state.locked === owner) { "Mutex is locked by ${state.locked} but expected $owner" } - if (_state.compareAndSet(state, EMPTY_UNLOCKED)) return - } - is OpDescriptor -> state.perform(this) - is LockedQueue -> { - if (owner != null) - check(state.owner === owner) { "Mutex is locked by ${state.owner} but expected $owner" } - val waiter = state.removeFirstOrNull() - if (waiter == null) { - val op = UnlockOp(state) - if (_state.compareAndSet(state, op) && op.perform(this) == null) return - } else { - if ((waiter as LockWaiter).tryResumeLockWaiter()) { - state.owner = waiter.owner ?: LOCKED - waiter.completeResumeLockWaiter() - return - } - } - } - else -> error("Illegal state $state") - } + while (true) { + // Is this mutex locked? + check(isLocked) { "This mutex is not locked" } + // Read the owner, waiting until it is set in a spin-loop if required. + val curOwner = this.owner.value + if (curOwner === NO_OWNER) continue // <-- ATTENTION, BLOCKING PART HERE + // Check the owner. + check(curOwner === owner || owner == null) { "This mutex is locked by $curOwner, but $owner is expected" } + // Try to clean the owner first. We need to use CAS here to synchronize with concurrent `unlock(..)`-s. + if (!this.owner.compareAndSet(curOwner, NO_OWNER)) continue + // Release the semaphore permit at the end. + release() + return } } - override fun toString(): String { - _state.loop { state -> - when (state) { - is Empty -> return "Mutex[${state.locked}]" - is OpDescriptor -> state.perform(this) - is LockedQueue -> return "Mutex[${state.owner}]" - else -> error("Illegal state $state") - } + @Suppress("UNCHECKED_CAST", "OverridingDeprecatedMember", "OVERRIDE_DEPRECATION") + override val onLock: SelectClause2<Any?, Mutex> get() = SelectClause2Impl( + clauseObject = this, + regFunc = MutexImpl::onLockRegFunction as RegistrationFunction, + processResFunc = MutexImpl::onLockProcessResult as ProcessResultFunction, + onCancellationConstructor = onSelectCancellationUnlockConstructor + ) + + protected open fun onLockRegFunction(select: SelectInstance<*>, owner: Any?) { + if (owner != null && holdsLock(owner)) { + select.selectInRegistrationPhase(ON_LOCK_ALREADY_LOCKED_BY_OWNER) + } else { + onAcquireRegFunction(SelectInstanceWithOwner(select as SelectInstanceInternal<*>, owner), owner) } } - private class LockedQueue( - @Volatile @JvmField var owner: Any - ) : LockFreeLinkedListHead() { - override fun toString(): String = "LockedQueue[$owner]" - } - - private abstract inner class LockWaiter( - @JvmField val owner: Any? - ) : LockFreeLinkedListNode(), DisposableHandle { - private val isTaken = atomic(false) - fun take(): Boolean = isTaken.compareAndSet(false, true) - final override fun dispose() { remove() } - abstract fun tryResumeLockWaiter(): Boolean - abstract fun completeResumeLockWaiter() + protected open fun onLockProcessResult(owner: Any?, result: Any?): Any? { + if (result == ON_LOCK_ALREADY_LOCKED_BY_OWNER) { + error("This mutex is already locked by the specified owner: $owner") + } + return this } - private inner class LockCont( - owner: Any?, - private val cont: CancellableContinuation<Unit> - ) : LockWaiter(owner) { - - override fun tryResumeLockWaiter(): Boolean { - if (!take()) return false - return cont.tryResume(Unit, idempotent = null) { - // if this continuation gets cancelled during dispatch to the caller, then release the lock + private inner class CancellableContinuationWithOwner( + @JvmField + val cont: CancellableContinuationImpl<Unit>, + @JvmField + val owner: Any? + ) : CancellableContinuation<Unit> by cont, Waiter by cont { + override fun tryResume(value: Unit, idempotent: Any?, onCancellation: ((cause: Throwable) -> Unit)?): Any? { + assert { this@MutexImpl.owner.value === NO_OWNER } + val token = cont.tryResume(value, idempotent) { + assert { this@MutexImpl.owner.value.let { it === NO_OWNER ||it === owner } } + this@MutexImpl.owner.value = owner unlock(owner) - } != null + } + if (token != null) { + assert { this@MutexImpl.owner.value === NO_OWNER } + this@MutexImpl.owner.value = owner + } + return token } - override fun completeResumeLockWaiter() = cont.completeResume(RESUME_TOKEN) - override fun toString(): String = "LockCont[$owner, ${cont}] for ${this@MutexImpl}" + override fun resume(value: Unit, onCancellation: ((cause: Throwable) -> Unit)?) { + assert { this@MutexImpl.owner.value === NO_OWNER } + this@MutexImpl.owner.value = owner + cont.resume(value) { unlock(owner) } + } } - private inner class LockSelect<R>( - owner: Any?, - @JvmField val select: SelectInstance<R>, - @JvmField val block: suspend (Mutex) -> R - ) : LockWaiter(owner) { - override fun tryResumeLockWaiter(): Boolean = take() && select.trySelect() - override fun completeResumeLockWaiter() { - block.startCoroutineCancellable(receiver = this@MutexImpl, completion = select.completion) { - // if this continuation gets cancelled during dispatch to the caller, then release the lock - unlock(owner) + private inner class SelectInstanceWithOwner<Q>( + @JvmField + val select: SelectInstanceInternal<Q>, + @JvmField + val owner: Any? + ) : SelectInstanceInternal<Q> by select { + override fun trySelect(clauseObject: Any, result: Any?): Boolean { + assert { this@MutexImpl.owner.value === NO_OWNER } + return select.trySelect(clauseObject, result).also { success -> + if (success) this@MutexImpl.owner.value = owner } } - override fun toString(): String = "LockSelect[$owner, $select] for ${this@MutexImpl}" - } - - // atomic unlock operation that checks that waiters queue is empty - private class UnlockOp( - @JvmField val queue: LockedQueue - ) : AtomicOp<MutexImpl>() { - override fun prepare(affected: MutexImpl): Any? = - if (queue.isEmpty) null else UNLOCK_FAIL - override fun complete(affected: MutexImpl, failure: Any?) { - val update: Any = if (failure == null) EMPTY_UNLOCKED else queue - affected._state.compareAndSet(this, update) + override fun selectInRegistrationPhase(internalResult: Any?) { + assert { this@MutexImpl.owner.value === NO_OWNER } + this@MutexImpl.owner.value = owner + select.selectInRegistrationPhase(internalResult) } } + + override fun toString() = "Mutex@${hexAddress}[isLocked=$isLocked,owner=${owner.value}]" } + +private val NO_OWNER = Symbol("NO_OWNER") +private val ON_LOCK_ALREADY_LOCKED_BY_OWNER = Symbol("ALREADY_LOCKED_BY_OWNER") + +private const val TRY_LOCK_SUCCESS = 0 +private const val TRY_LOCK_FAILED = 1 +private const val TRY_LOCK_ALREADY_LOCKED_BY_OWNER = 2 + +private const val HOLDS_LOCK_UNLOCKED = 0 +private const val HOLDS_LOCK_YES = 1 +private const val HOLDS_LOCK_ANOTHER_OWNER = 2 diff --git a/kotlinx-coroutines-core/common/src/sync/Semaphore.kt b/kotlinx-coroutines-core/common/src/sync/Semaphore.kt index e8b28bc1..9f30721d 100644 --- a/kotlinx-coroutines-core/common/src/sync/Semaphore.kt +++ b/kotlinx-coroutines-core/common/src/sync/Semaphore.kt @@ -7,10 +7,11 @@ package kotlinx.coroutines.sync import kotlinx.atomicfu.* import kotlinx.coroutines.* import kotlinx.coroutines.internal.* +import kotlinx.coroutines.selects.* import kotlin.contracts.* import kotlin.coroutines.* +import kotlin.js.* import kotlin.math.* -import kotlin.native.concurrent.SharedImmutable /** * A counting semaphore for coroutines that logically maintains a number of available permits. @@ -18,7 +19,7 @@ import kotlin.native.concurrent.SharedImmutable * Each [release] adds a permit, potentially releasing a suspended acquirer. * Semaphore is fair and maintains a FIFO order of acquirers. * - * Semaphores are mostly used to limit the number of coroutines that have an access to particular resource. + * Semaphores are mostly used to limit the number of coroutines that have access to particular resource. * Semaphore with `permits = 1` is essentially a [Mutex]. **/ public interface Semaphore { @@ -42,7 +43,7 @@ public interface Semaphore { * Use [CoroutineScope.isActive] or [CoroutineScope.ensureActive] to periodically * check for cancellation in tight loops if needed. * - * Use [tryAcquire] to try acquire a permit of this semaphore without suspension. + * Use [tryAcquire] to try to acquire a permit of this semaphore without suspension. */ public suspend fun acquire() @@ -90,7 +91,8 @@ public suspend inline fun <T> Semaphore.withPermit(action: () -> T): T { } } -private class SemaphoreImpl(private val permits: Int, acquiredPermits: Int) : Semaphore { +@Suppress("UNCHECKED_CAST") +internal open class SemaphoreImpl(private val permits: Int, acquiredPermits: Int) : Semaphore { /* The queue of waiting acquirers is essentially an infinite array based on the list of segments (see `SemaphoreSegment`); each segment contains a fixed number of slots. To determine a slot for each enqueue @@ -140,11 +142,11 @@ private class SemaphoreImpl(private val permits: Int, acquiredPermits: Int) : Se } /** - * This counter indicates a number of available permits if it is non-negative, - * or the size with minus sign otherwise. Note, that 32-bit counter is enough here - * since the maximal number of available permits is [permits] which is [Int], - * and the maximum number of waiting acquirers cannot be greater than 2^31 in any - * real application. + * This counter indicates the number of available permits if it is positive, + * or the negated number of waiters on this semaphore otherwise. + * Note, that 32-bit counter is enough here since the maximal number of available + * permits is [permits] which is [Int], and the maximum number of waiting acquirers + * cannot be greater than 2^31 in any real application. */ private val _availablePermits = atomic(permits - acquiredPermits) override val availablePermits: Int get() = max(_availablePermits.value, 0) @@ -152,62 +154,160 @@ private class SemaphoreImpl(private val permits: Int, acquiredPermits: Int) : Se private val onCancellationRelease = { _: Throwable -> release() } override fun tryAcquire(): Boolean { - _availablePermits.loop { p -> + while (true) { + // Get the current number of available permits. + val p = _availablePermits.value + // Is the number of available permits greater + // than the maximal one because of an incorrect + // `release()` call without a preceding `acquire()`? + // Change it to `permits` and start from the beginning. + if (p > permits) { + coerceAvailablePermitsAtMaximum() + continue + } + // Try to decrement the number of available + // permits if it is greater than zero. if (p <= 0) return false if (_availablePermits.compareAndSet(p, p - 1)) return true } } override suspend fun acquire() { - val p = _availablePermits.getAndDecrement() + // Decrement the number of available permits. + val p = decPermits() + // Is the permit acquired? if (p > 0) return // permit acquired + // Try to suspend otherwise. // While it looks better when the following function is inlined, // it is important to make `suspend` function invocations in a way - // so that the tail-call optimization can be applied. + // so that the tail-call optimization can be applied here. acquireSlowPath() } private suspend fun acquireSlowPath() = suspendCancellableCoroutineReusable<Unit> sc@ { cont -> + // Try to suspend. + if (addAcquireToQueue(cont)) return@sc + // The suspension has been failed + // due to the synchronous resumption mode. + // Restart the whole `acquire`. + acquire(cont) + } + + @JsName("acquireCont") + protected fun acquire(waiter: CancellableContinuation<Unit>) = acquire( + waiter = waiter, + suspend = { cont -> addAcquireToQueue(cont as Waiter) }, + onAcquired = { cont -> cont.resume(Unit, onCancellationRelease) } + ) + + @JsName("acquireInternal") + private inline fun <W> acquire(waiter: W, suspend: (waiter: W) -> Boolean, onAcquired: (waiter: W) -> Unit) { while (true) { - if (addAcquireToQueue(cont)) return@sc - val p = _availablePermits.getAndDecrement() - if (p > 0) { // permit acquired - cont.resume(Unit, onCancellationRelease) - return@sc + // Decrement the number of available permits at first. + val p = decPermits() + // Is the permit acquired? + if (p > 0) { + onAcquired(waiter) + return } + // Permit has not been acquired, try to suspend. + if (suspend(waiter)) return + } + } + + // We do not fully support `onAcquire` as it is needed only for `Mutex.onLock`. + @Suppress("UNUSED_PARAMETER") + protected fun onAcquireRegFunction(select: SelectInstance<*>, ignoredParam: Any?) = + acquire( + waiter = select, + suspend = { s -> addAcquireToQueue(s as Waiter) }, + onAcquired = { s -> s.selectInRegistrationPhase(Unit) } + ) + + /** + * Decrements the number of available permits + * and ensures that it is not greater than [permits] + * at the point of decrement. The last may happen + * due to an incorrect `release()` call without + * a preceding `acquire()`. + */ + private fun decPermits(): Int { + while (true) { + // Decrement the number of available permits. + val p = _availablePermits.getAndDecrement() + // Is the number of available permits greater + // than the maximal one due to an incorrect + // `release()` call without a preceding `acquire()`? + if (p > permits) continue + // The number of permits is correct, return it. + return p } } override fun release() { while (true) { - val p = _availablePermits.getAndUpdate { cur -> - check(cur < permits) { "The number of released permits cannot be greater than $permits" } - cur + 1 + // Increment the number of available permits. + val p = _availablePermits.getAndIncrement() + // Is this `release` call correct and does not + // exceed the maximal number of permits? + if (p >= permits) { + // Revert the number of available permits + // back to the correct one and fail with error. + coerceAvailablePermitsAtMaximum() + error("The number of released permits cannot be greater than $permits") } + // Is there a waiter that should be resumed? if (p >= 0) return + // Try to resume the first waiter, and + // restart the operation if either this + // first waiter is cancelled or + // due to `SYNC` resumption mode. if (tryResumeNextFromQueue()) return } } /** + * Changes the number of available permits to + * [permits] if it became greater due to an + * incorrect [release] call. + */ + private fun coerceAvailablePermitsAtMaximum() { + while (true) { + val cur = _availablePermits.value + if (cur <= permits) break + if (_availablePermits.compareAndSet(cur, permits)) break + } + } + + /** * Returns `false` if the received permit cannot be used and the calling operation should restart. */ - private fun addAcquireToQueue(cont: CancellableContinuation<Unit>): Boolean { + private fun addAcquireToQueue(waiter: Waiter): Boolean { val curTail = this.tail.value val enqIdx = enqIdx.getAndIncrement() + val createNewSegment = ::createSegment val segment = this.tail.findSegmentAndMoveForward(id = enqIdx / SEGMENT_SIZE, startFrom = curTail, - createNewSegment = ::createSegment).segment // cannot be closed + createNewSegment = createNewSegment).segment // cannot be closed val i = (enqIdx % SEGMENT_SIZE).toInt() // the regular (fast) path -- if the cell is empty, try to install continuation - if (segment.cas(i, null, cont)) { // installed continuation successfully - cont.invokeOnCancellation(CancelSemaphoreAcquisitionHandler(segment, i).asHandler) + if (segment.cas(i, null, waiter)) { // installed continuation successfully + waiter.invokeOnCancellation(segment, i) return true } // On CAS failure -- the cell must be either PERMIT or BROKEN // If the cell already has PERMIT from tryResumeNextFromQueue, try to grab it if (segment.cas(i, PERMIT, TAKEN)) { // took permit thus eliminating acquire/release pair /// This continuation is not yet published, but still can be cancelled via outer job - cont.resume(Unit, onCancellationRelease) + when (waiter) { + is CancellableContinuation<*> -> { + waiter as CancellableContinuation<Unit> + waiter.resume(Unit, onCancellationRelease) + } + is SelectInstance<*> -> { + waiter.selectInRegistrationPhase(Unit) + } + else -> error("unexpected: $waiter") + } return true } assert { segment.get(i) === BROKEN } // it must be broken in this case, no other way around it @@ -219,8 +319,9 @@ private class SemaphoreImpl(private val permits: Int, acquiredPermits: Int) : Se val curHead = this.head.value val deqIdx = deqIdx.getAndIncrement() val id = deqIdx / SEGMENT_SIZE + val createNewSegment = ::createSegment val segment = this.head.findSegmentAndMoveForward(id, startFrom = curHead, - createNewSegment = ::createSegment).segment // cannot be closed + createNewSegment = createNewSegment).segment // cannot be closed segment.cleanPrev() if (segment.id > id) return false val i = (deqIdx % SEGMENT_SIZE).toInt() @@ -235,34 +336,32 @@ private class SemaphoreImpl(private val permits: Int, acquiredPermits: Int) : Se // Try to break the slot in order not to wait return !segment.cas(i, PERMIT, BROKEN) } - cellState === CANCELLED -> return false // the acquire was already cancelled - else -> return (cellState as CancellableContinuation<Unit>).tryResumeAcquire() + cellState === CANCELLED -> return false // the acquirer has already been cancelled + else -> return cellState.tryResumeAcquire() } } - private fun CancellableContinuation<Unit>.tryResumeAcquire(): Boolean { - val token = tryResume(Unit, null, onCancellationRelease) ?: return false - completeResume(token) - return true - } -} - -private class CancelSemaphoreAcquisitionHandler( - private val segment: SemaphoreSegment, - private val index: Int -) : CancelHandler() { - override fun invoke(cause: Throwable?) { - segment.cancel(index) + private fun Any.tryResumeAcquire(): Boolean = when(this) { + is CancellableContinuation<*> -> { + this as CancellableContinuation<Unit> + val token = tryResume(Unit, null, onCancellationRelease) + if (token != null) { + completeResume(token) + true + } else false + } + is SelectInstance<*> -> { + trySelect(this@SemaphoreImpl, Unit) + } + else -> error("unexpected: $this") } - - override fun toString() = "CancelSemaphoreAcquisitionHandler[$segment, $index]" } private fun createSegment(id: Long, prev: SemaphoreSegment?) = SemaphoreSegment(id, prev, 0) private class SemaphoreSegment(id: Long, prev: SemaphoreSegment?, pointers: Int) : Segment<SemaphoreSegment>(id, prev, pointers) { val acquirers = atomicArrayOfNulls<Any?>(SEGMENT_SIZE) - override val maxSlots: Int get() = SEGMENT_SIZE + override val numberOfSlots: Int get() = SEGMENT_SIZE @Suppress("NOTHING_TO_INLINE") inline fun get(index: Int): Any? = acquirers[index].value @@ -280,7 +379,7 @@ private class SemaphoreSegment(id: Long, prev: SemaphoreSegment?, pointers: Int) // Cleans the acquirer slot located by the specified index // and removes this segment physically if all slots are cleaned. - fun cancel(index: Int) { + override fun onCancellation(index: Int, cause: Throwable?, context: CoroutineContext) { // Clean the slot set(index, CANCELLED) // Remove this segment if needed @@ -289,15 +388,9 @@ private class SemaphoreSegment(id: Long, prev: SemaphoreSegment?, pointers: Int) override fun toString() = "SemaphoreSegment[id=$id, hashCode=${hashCode()}]" } -@SharedImmutable private val MAX_SPIN_CYCLES = systemProp("kotlinx.coroutines.semaphore.maxSpinCycles", 100) -@SharedImmutable private val PERMIT = Symbol("PERMIT") -@SharedImmutable private val TAKEN = Symbol("TAKEN") -@SharedImmutable private val BROKEN = Symbol("BROKEN") -@SharedImmutable private val CANCELLED = Symbol("CANCELLED") -@SharedImmutable private val SEGMENT_SIZE = systemProp("kotlinx.coroutines.semaphore.segmentSize", 16) diff --git a/kotlinx-coroutines-core/common/test/CancellableContinuationHandlersTest.kt b/kotlinx-coroutines-core/common/test/CancellableContinuationHandlersTest.kt index 3c11182e..54bc18c1 100644 --- a/kotlinx-coroutines-core/common/test/CancellableContinuationHandlersTest.kt +++ b/kotlinx-coroutines-core/common/test/CancellableContinuationHandlersTest.kt @@ -6,6 +6,7 @@ package kotlinx.coroutines +import kotlinx.coroutines.internal.* import kotlin.coroutines.* import kotlin.test.* @@ -159,4 +160,31 @@ class CancellableContinuationHandlersTest : TestBase() { } finish(3) } + + @Test + fun testSegmentAsHandler() = runTest { + class MySegment : Segment<MySegment>(0, null, 0) { + override val numberOfSlots: Int get() = 0 + + var invokeOnCancellationCalled = false + override fun onCancellation(index: Int, cause: Throwable?, context: CoroutineContext) { + invokeOnCancellationCalled = true + } + } + val s = MySegment() + expect(1) + try { + suspendCancellableCoroutine<Unit> { c -> + expect(2) + c as CancellableContinuationImpl<*> + c.invokeOnCancellation(s, 0) + c.cancel() + } + } catch (e: CancellationException) { + expect(3) + } + expect(4) + check(s.invokeOnCancellationCalled) + finish(5) + } } diff --git a/kotlinx-coroutines-core/common/test/CoroutineScopeTest.kt b/kotlinx-coroutines-core/common/test/CoroutineScopeTest.kt index c46f41a0..b678b03c 100644 --- a/kotlinx-coroutines-core/common/test/CoroutineScopeTest.kt +++ b/kotlinx-coroutines-core/common/test/CoroutineScopeTest.kt @@ -277,4 +277,15 @@ class CoroutineScopeTest : TestBase() { private fun scopePlusContext(c1: CoroutineContext, c2: CoroutineContext) = (ContextScope(c1) + c2).coroutineContext + + @Test + fun testIsActiveWithoutJob() { + var invoked = false + suspend fun testIsActive() { + assertTrue(coroutineContext.isActive) + invoked = true + } + ::testIsActive.startCoroutine(Continuation(EmptyCoroutineContext){}) + assertTrue(invoked) + } } diff --git a/kotlinx-coroutines-core/common/test/JobStatesTest.kt b/kotlinx-coroutines-core/common/test/JobStatesTest.kt index dfcb462c..739401f6 100644 --- a/kotlinx-coroutines-core/common/test/JobStatesTest.kt +++ b/kotlinx-coroutines-core/common/test/JobStatesTest.kt @@ -16,6 +16,7 @@ class JobStatesTest : TestBase() { @Test public fun testNormalCompletion() = runTest { expect(1) + val parent = coroutineContext.job val job = launch(start = CoroutineStart.LAZY) { expect(2) // launches child @@ -28,23 +29,27 @@ class JobStatesTest : TestBase() { assertFalse(job.isActive) assertFalse(job.isCompleted) assertFalse(job.isCancelled) + assertSame(parent, job.parent) // New -> Active job.start() assertTrue(job.isActive) assertFalse(job.isCompleted) assertFalse(job.isCancelled) + assertSame(parent, job.parent) // Active -> Completing yield() // scheduled & starts child expect(3) assertTrue(job.isActive) assertFalse(job.isCompleted) assertFalse(job.isCancelled) + assertSame(parent, job.parent) // Completing -> Completed yield() finish(5) assertFalse(job.isActive) assertTrue(job.isCompleted) assertFalse(job.isCancelled) + assertNull(job.parent) } @Test @@ -159,4 +164,4 @@ class JobStatesTest : TestBase() { assertTrue(job.isCompleted) assertTrue(job.isCancelled) } -}
\ No newline at end of file +} diff --git a/kotlinx-coroutines-core/common/test/JobTest.kt b/kotlinx-coroutines-core/common/test/JobTest.kt index 04d3c9e0..38895830 100644 --- a/kotlinx-coroutines-core/common/test/JobTest.kt +++ b/kotlinx-coroutines-core/common/test/JobTest.kt @@ -12,6 +12,7 @@ class JobTest : TestBase() { @Test fun testState() { val job = Job() + assertNull(job.parent) assertTrue(job.isActive) job.cancel() assertTrue(!job.isActive) @@ -102,11 +103,11 @@ class JobTest : TestBase() { } assertTrue(job.isActive) for (i in 0 until n) assertEquals(0, fireCount[i]) - val tryCancel = Try { job.cancel() } + val cancelResult = runCatching { job.cancel() } assertTrue(!job.isActive) for (i in 0 until n) assertEquals(1, fireCount[i]) - assertTrue(tryCancel.exception is CompletionHandlerException) - assertTrue(tryCancel.exception!!.cause is TestException) + assertTrue(cancelResult.exceptionOrNull() is CompletionHandlerException) + assertTrue(cancelResult.exceptionOrNull()!!.cause is TestException) } @Test @@ -210,11 +211,13 @@ class JobTest : TestBase() { @Test fun testIncompleteJobState() = runTest { + val parent = coroutineContext.job val job = launch { coroutineContext[Job]!!.invokeOnCompletion { } } - + assertSame(parent, job.parent) job.join() + assertNull(job.parent) assertTrue(job.isCompleted) assertFalse(job.isActive) assertFalse(job.isCancelled) diff --git a/kotlinx-coroutines-core/common/test/TestBase.common.kt b/kotlinx-coroutines-core/common/test/TestBase.common.kt index 8b7024a6..06e71b45 100644 --- a/kotlinx-coroutines-core/common/test/TestBase.common.kt +++ b/kotlinx-coroutines-core/common/test/TestBase.common.kt @@ -104,3 +104,5 @@ class BadClass { override fun hashCode(): Int = error("hashCode") override fun toString(): String = error("toString") } + +public expect val isJavaAndWindows: Boolean diff --git a/kotlinx-coroutines-core/common/test/Try.kt b/kotlinx-coroutines-core/common/test/Try.kt deleted file mode 100644 index 194ea4ba..00000000 --- a/kotlinx-coroutines-core/common/test/Try.kt +++ /dev/null @@ -1,29 +0,0 @@ -/* - * Copyright 2016-2018 JetBrains s.r.o. Use of this source code is governed by the Apache 2.0 license. - */ - -package kotlinx.coroutines - -public class Try<out T> private constructor(private val _value: Any?) { - private class Fail(val exception: Throwable) { - override fun toString(): String = "Failure[$exception]" - } - - public companion object { - public operator fun <T> invoke(block: () -> T): Try<T> = - try { - Success(block()) - } catch(e: Throwable) { - Failure<T>(e) - } - public fun <T> Success(value: T) = Try<T>(value) - public fun <T> Failure(exception: Throwable) = Try<T>(Fail(exception)) - } - - @Suppress("UNCHECKED_CAST") - public val value: T get() = if (_value is Fail) throw _value.exception else _value as T - - public val exception: Throwable? get() = (_value as? Fail)?.exception - - override fun toString(): String = _value.toString() -} diff --git a/kotlinx-coroutines-core/common/test/WithTimeoutOrNullDurationTest.kt b/kotlinx-coroutines-core/common/test/WithTimeoutOrNullDurationTest.kt index 92dba7b3..1f9ad46f 100644 --- a/kotlinx-coroutines-core/common/test/WithTimeoutOrNullDurationTest.kt +++ b/kotlinx-coroutines-core/common/test/WithTimeoutOrNullDurationTest.kt @@ -131,6 +131,7 @@ class WithTimeoutOrNullDurationTest : TestBase() { @Test fun testOuterTimeout() = runTest { + if (isJavaAndWindows) return@runTest var counter = 0 val result = withTimeoutOrNull(320.milliseconds) { while (true) { diff --git a/kotlinx-coroutines-core/common/test/WithTimeoutOrNullTest.kt b/kotlinx-coroutines-core/common/test/WithTimeoutOrNullTest.kt index ee896c9b..5ab8ae7d 100644 --- a/kotlinx-coroutines-core/common/test/WithTimeoutOrNullTest.kt +++ b/kotlinx-coroutines-core/common/test/WithTimeoutOrNullTest.kt @@ -128,6 +128,7 @@ class WithTimeoutOrNullTest : TestBase() { @Test fun testOuterTimeout() = runTest { + if (isJavaAndWindows) return@runTest var counter = 0 val result = withTimeoutOrNull(320) { while (true) { diff --git a/kotlinx-coroutines-core/common/test/channels/BasicOperationsTest.kt b/kotlinx-coroutines-core/common/test/channels/BasicOperationsTest.kt index 4538f6c6..1f9b6fa7 100644 --- a/kotlinx-coroutines-core/common/test/channels/BasicOperationsTest.kt +++ b/kotlinx-coroutines-core/common/test/channels/BasicOperationsTest.kt @@ -21,7 +21,7 @@ class BasicOperationsTest : TestBase() { @Test fun testTrySendAfterClose() = runTest { - TestChannelKind.values().forEach { kind -> testTrySend(kind) } + TestChannelKind.values().forEach { kind -> testTrySendAfterClose(kind) } } @Test @@ -85,6 +85,45 @@ class BasicOperationsTest : TestBase() { } } + @Test + fun testCancelledChannelInvokeOnClose() { + val ch = Channel<Int>() + ch.invokeOnClose { assertIs<CancellationException>(it) } + ch.cancel() + } + + @Test + fun testCancelledChannelWithCauseInvokeOnClose() { + val ch = Channel<Int>() + ch.invokeOnClose { assertIs<TimeoutCancellationException>(it) } + ch.cancel(TimeoutCancellationException("")) + } + + @Test + fun testThrowingInvokeOnClose() = runTest { + val channel = Channel<Int>() + channel.invokeOnClose { + assertNull(it) + expect(3) + throw TestException() + } + + launch { + try { + expect(2) + channel.close() + } catch (e: TestException) { + expect(4) + } + } + expect(1) + yield() + assertTrue(channel.isClosedForReceive) + assertTrue(channel.isClosedForSend) + assertFalse(channel.close()) + finish(5) + } + @Suppress("ReplaceAssertBooleanWithAssertEquality") private suspend fun testReceiveCatching(kind: TestChannelKind) = coroutineScope { reset() @@ -114,7 +153,7 @@ class BasicOperationsTest : TestBase() { finish(6) } - private suspend fun testTrySend(kind: TestChannelKind) = coroutineScope { + private suspend fun testTrySendAfterClose(kind: TestChannelKind) = coroutineScope { val channel = kind.create<Int>() val d = async { channel.send(42) } yield() @@ -124,7 +163,7 @@ class BasicOperationsTest : TestBase() { channel.trySend(2) .onSuccess { expectUnreached() } .onClosed { - assertTrue { it is ClosedSendChannelException} + assertTrue { it is ClosedSendChannelException } if (!kind.isConflated) { assertEquals(42, channel.receive()) } diff --git a/kotlinx-coroutines-core/common/test/channels/BroadcastChannelFactoryTest.kt b/kotlinx-coroutines-core/common/test/channels/BroadcastChannelFactoryTest.kt index 61e93fa8..e27edcf6 100644 --- a/kotlinx-coroutines-core/common/test/channels/BroadcastChannelFactoryTest.kt +++ b/kotlinx-coroutines-core/common/test/channels/BroadcastChannelFactoryTest.kt @@ -16,7 +16,7 @@ class BroadcastChannelFactoryTest : TestBase() { } @Test - fun testLinkedListChannelNotSupported() { + fun testUnlimitedChannelNotSupported() { assertFailsWith<IllegalArgumentException> { BroadcastChannel<Int>(Channel.UNLIMITED) } } @@ -26,9 +26,9 @@ class BroadcastChannelFactoryTest : TestBase() { } @Test - fun testArrayBroadcastChannel() { - assertTrue { BroadcastChannel<Int>(1) is ArrayBroadcastChannel } - assertTrue { BroadcastChannel<Int>(10) is ArrayBroadcastChannel } + fun testBufferedBroadcastChannel() { + assertTrue { BroadcastChannel<Int>(1) is BroadcastChannelImpl } + assertTrue { BroadcastChannel<Int>(10) is BroadcastChannelImpl } } @Test diff --git a/kotlinx-coroutines-core/common/test/channels/BroadcastTest.kt b/kotlinx-coroutines-core/common/test/channels/BroadcastTest.kt index ab1a85d6..34b13955 100644 --- a/kotlinx-coroutines-core/common/test/channels/BroadcastTest.kt +++ b/kotlinx-coroutines-core/common/test/channels/BroadcastTest.kt @@ -7,6 +7,7 @@ package kotlinx.coroutines.channels import kotlinx.coroutines.* +import kotlinx.coroutines.selects.* import kotlin.test.* class BroadcastTest : TestBase() { @@ -17,7 +18,7 @@ class BroadcastTest : TestBase() { expect(4) send(1) // goes to receiver expect(5) - send(2) // goes to buffer + select<Unit> { onSend(2) {} } // goes to buffer expect(6) send(3) // suspends, will not be consumes, but will not be cancelled either expect(10) diff --git a/kotlinx-coroutines-core/common/test/channels/ArrayBroadcastChannelTest.kt b/kotlinx-coroutines-core/common/test/channels/BufferedBroadcastChannelTest.kt index 2d71cc94..fad65008 100644 --- a/kotlinx-coroutines-core/common/test/channels/ArrayBroadcastChannelTest.kt +++ b/kotlinx-coroutines-core/common/test/channels/BufferedBroadcastChannelTest.kt @@ -7,7 +7,7 @@ package kotlinx.coroutines.channels import kotlinx.coroutines.* import kotlin.test.* -class ArrayBroadcastChannelTest : TestBase() { +class BufferedBroadcastChannelTest : TestBase() { @Test fun testConcurrentModification() = runTest { diff --git a/kotlinx-coroutines-core/common/test/channels/ArrayChannelTest.kt b/kotlinx-coroutines-core/common/test/channels/BufferedChannelTest.kt index 632fd292..0f703521 100644 --- a/kotlinx-coroutines-core/common/test/channels/ArrayChannelTest.kt +++ b/kotlinx-coroutines-core/common/test/channels/BufferedChannelTest.kt @@ -7,7 +7,7 @@ package kotlinx.coroutines.channels import kotlinx.coroutines.* import kotlin.test.* -class ArrayChannelTest : TestBase() { +class BufferedChannelTest : TestBase() { @Test fun testSimple() = runTest { val q = Channel<Int>(1) @@ -34,6 +34,7 @@ class ArrayChannelTest : TestBase() { sender.join() receiver.join() check(q.isEmpty) + (q as BufferedChannel<*>).checkSegmentStructureInvariants() finish(10) } @@ -59,6 +60,7 @@ class ArrayChannelTest : TestBase() { check(!q.isEmpty && q.isClosedForSend && !q.isClosedForReceive) yield() check(!q.isEmpty && q.isClosedForSend && q.isClosedForReceive) + (q as BufferedChannel<*>).checkSegmentStructureInvariants() finish(8) } @@ -81,6 +83,7 @@ class ArrayChannelTest : TestBase() { expect(6) try { q.send(42) } catch (e: ClosedSendChannelException) { + (q as BufferedChannel<*>).checkSegmentStructureInvariants() finish(7) } } @@ -112,6 +115,7 @@ class ArrayChannelTest : TestBase() { expect(8) assertFalse(q.trySend(4).isSuccess) yield() + (q as BufferedChannel<*>).checkSegmentStructureInvariants() finish(12) } @@ -135,6 +139,7 @@ class ArrayChannelTest : TestBase() { check(q.isClosedForSend) check(q.isClosedForReceive) assertFailsWith<CancellationException> { q.receiveCatching().getOrThrow() } + (q as BufferedChannel<*>).checkSegmentStructureInvariants() finish(12) } @@ -165,6 +170,11 @@ class ArrayChannelTest : TestBase() { checkBufferChannel(channel, capacity) } + @Test + fun testBufferIsNotPreallocated() { + (0..100_000).map { Channel<Int>(Int.MAX_VALUE / 2) } + } + private suspend fun CoroutineScope.checkBufferChannel( channel: Channel<Int>, capacity: Int @@ -189,6 +199,7 @@ class ArrayChannelTest : TestBase() { result.add(it) } assertEquals((0..capacity).toList(), result) + (channel as BufferedChannel<*>).checkSegmentStructureInvariants() finish(6) } } diff --git a/kotlinx-coroutines-core/common/test/channels/ChannelFactoryTest.kt b/kotlinx-coroutines-core/common/test/channels/ChannelFactoryTest.kt index 413c91f5..706a2fdd 100644 --- a/kotlinx-coroutines-core/common/test/channels/ChannelFactoryTest.kt +++ b/kotlinx-coroutines-core/common/test/channels/ChannelFactoryTest.kt @@ -11,28 +11,28 @@ import kotlin.test.* class ChannelFactoryTest : TestBase() { @Test fun testRendezvousChannel() { - assertTrue(Channel<Int>() is RendezvousChannel) - assertTrue(Channel<Int>(0) is RendezvousChannel) + assertTrue(Channel<Int>() is BufferedChannel) + assertTrue(Channel<Int>(0) is BufferedChannel) } @Test - fun testLinkedListChannel() { - assertTrue(Channel<Int>(Channel.UNLIMITED) is LinkedListChannel) - assertTrue(Channel<Int>(Channel.UNLIMITED, BufferOverflow.DROP_OLDEST) is LinkedListChannel) - assertTrue(Channel<Int>(Channel.UNLIMITED, BufferOverflow.DROP_LATEST) is LinkedListChannel) + fun testUnlimitedChannel() { + assertTrue(Channel<Int>(Channel.UNLIMITED) is BufferedChannel) + assertTrue(Channel<Int>(Channel.UNLIMITED, BufferOverflow.DROP_OLDEST) is BufferedChannel) + assertTrue(Channel<Int>(Channel.UNLIMITED, BufferOverflow.DROP_LATEST) is BufferedChannel) } @Test fun testConflatedChannel() { - assertTrue(Channel<Int>(Channel.CONFLATED) is ConflatedChannel) - assertTrue(Channel<Int>(1, BufferOverflow.DROP_OLDEST) is ConflatedChannel) + assertTrue(Channel<Int>(Channel.CONFLATED) is ConflatedBufferedChannel) + assertTrue(Channel<Int>(1, BufferOverflow.DROP_OLDEST) is ConflatedBufferedChannel) } @Test - fun testArrayChannel() { - assertTrue(Channel<Int>(1) is ArrayChannel) - assertTrue(Channel<Int>(1, BufferOverflow.DROP_LATEST) is ArrayChannel) - assertTrue(Channel<Int>(10) is ArrayChannel) + fun testBufferedChannel() { + assertTrue(Channel<Int>(1) is BufferedChannel) + assertTrue(Channel<Int>(1, BufferOverflow.DROP_LATEST) is ConflatedBufferedChannel) + assertTrue(Channel<Int>(10) is BufferedChannel) } @Test diff --git a/kotlinx-coroutines-core/common/test/channels/ChannelUndeliveredElementFailureTest.kt b/kotlinx-coroutines-core/common/test/channels/ChannelUndeliveredElementFailureTest.kt index ae05fb8d..0faa79b3 100644 --- a/kotlinx-coroutines-core/common/test/channels/ChannelUndeliveredElementFailureTest.kt +++ b/kotlinx-coroutines-core/common/test/channels/ChannelUndeliveredElementFailureTest.kt @@ -107,11 +107,70 @@ class ChannelUndeliveredElementFailureTest : TestBase() { } @Test - fun testChannelCancelledFail() = runTest(expected = { it.isElementCancelException()}) { + fun testChannelCancelledFail() = runTest(expected = { it.isElementCancelException() }) { val channel = Channel(1, onUndeliveredElement = onCancelFail) channel.send(item) channel.cancel() expectUnreached() } + @Test + fun testFailedHandlerInClosedConflatedChannel() = runTest(expected = { it is UndeliveredElementException }) { + val conflated = Channel<Int>(Channel.CONFLATED, onUndeliveredElement = { + finish(2) + throw TestException() + }) + expect(1) + conflated.close(IndexOutOfBoundsException()) + conflated.send(3) + } + + @Test + fun testFailedHandlerInClosedBufferedChannel() = runTest(expected = { it is UndeliveredElementException }) { + val conflated = Channel<Int>(3, onUndeliveredElement = { + finish(2) + throw TestException() + }) + expect(1) + conflated.close(IndexOutOfBoundsException()) + conflated.send(3) + } + + @Test + fun testSendDropOldestInvokeHandlerBuffered() = runTest(expected = { it is UndeliveredElementException }) { + val channel = Channel<Int>(1, BufferOverflow.DROP_OLDEST, onUndeliveredElement = { + finish(2) + throw TestException() + }) + + channel.send(42) + expect(1) + channel.send(12) + } + + @Test + fun testSendDropLatestInvokeHandlerBuffered() = runTest(expected = { it is UndeliveredElementException }) { + val channel = Channel<Int>(2, BufferOverflow.DROP_LATEST, onUndeliveredElement = { + finish(2) + throw TestException() + }) + + channel.send(42) + channel.send(12) + expect(1) + channel.send(12) + expectUnreached() + } + + @Test + fun testSendDropOldestInvokeHandlerConflated() = runTest(expected = { it is UndeliveredElementException }) { + val channel = Channel<Int>(Channel.CONFLATED, onUndeliveredElement = { + finish(2) + throw TestException() + }) + channel.send(42) + expect(1) + channel.send(42) + expectUnreached() + } } diff --git a/kotlinx-coroutines-core/common/test/channels/ChannelUndeliveredElementTest.kt b/kotlinx-coroutines-core/common/test/channels/ChannelUndeliveredElementTest.kt index f26361f2..3a994846 100644 --- a/kotlinx-coroutines-core/common/test/channels/ChannelUndeliveredElementTest.kt +++ b/kotlinx-coroutines-core/common/test/channels/ChannelUndeliveredElementTest.kt @@ -138,4 +138,63 @@ class ChannelUndeliveredElementTest : TestBase() { channel.send(Unit) finish(3) } + + @Test + fun testChannelBufferOverflow() = runTest { + testBufferOverflowStrategy(listOf(1, 2), BufferOverflow.DROP_OLDEST) + testBufferOverflowStrategy(listOf(3), BufferOverflow.DROP_LATEST) + } + + private suspend fun testBufferOverflowStrategy(expectedDroppedElements: List<Int>, strategy: BufferOverflow) { + val list = ArrayList<Int>() + val channel = Channel<Int>( + capacity = 2, + onBufferOverflow = strategy, + onUndeliveredElement = { value -> list.add(value) } + ) + + channel.send(1) + channel.send(2) + + channel.send(3) + channel.trySend(4).onFailure { expectUnreached() } + assertEquals(expectedDroppedElements, list) + } + + + @Test + fun testTrySendDoesNotInvokeHandlerOnClosedConflatedChannel() = runTest { + val conflated = Channel<Int>(Channel.CONFLATED, onUndeliveredElement = { + expectUnreached() + }) + conflated.close(IndexOutOfBoundsException()) + conflated.trySend(3) + } + + @Test + fun testTrySendDoesNotInvokeHandlerOnClosedChannel() = runTest { + val conflated = Channel<Int>(3, onUndeliveredElement = { + expectUnreached() + }) + conflated.close(IndexOutOfBoundsException()) + repeat(10) { + conflated.trySend(3) + } + } + + @Test + fun testTrySendDoesNotInvokeHandler() { + for (capacity in 0..2) { + testTrySendDoesNotInvokeHandler(capacity) + } + } + + private fun testTrySendDoesNotInvokeHandler(capacity: Int) { + val channel = Channel<Int>(capacity, BufferOverflow.DROP_LATEST, onUndeliveredElement = { + expectUnreached() + }) + repeat(10) { + channel.trySend(3) + } + } } diff --git a/kotlinx-coroutines-core/common/test/channels/ChannelsTest.kt b/kotlinx-coroutines-core/common/test/channels/ChannelsTest.kt index fb704c5b..e40071b9 100644 --- a/kotlinx-coroutines-core/common/test/channels/ChannelsTest.kt +++ b/kotlinx-coroutines-core/common/test/channels/ChannelsTest.kt @@ -20,7 +20,10 @@ class ChannelsTest: TestBase() { } @Test - fun testCloseWithMultipleWaiters() = runTest { + fun testCloseWithMultipleSuspendedReceivers() = runTest { + // Once the channel is closed, the waiting + // requests should be cancelled in the order + // they were suspended in the channel. val channel = Channel<Int>() launch { try { @@ -51,6 +54,40 @@ class ChannelsTest: TestBase() { } @Test + fun testCloseWithMultipleSuspendedSenders() = runTest { + // Once the channel is closed, the waiting + // requests should be cancelled in the order + // they were suspended in the channel. + val channel = Channel<Int>() + launch { + try { + expect(2) + channel.send(42) + expectUnreached() + } catch (e: CancellationException) { + expect(5) + } + } + + launch { + try { + expect(3) + channel.send(42) + expectUnreached() + } catch (e: CancellationException) { + expect(6) + } + } + + expect(1) + yield() + expect(4) + channel.cancel() + yield() + finish(7) + } + + @Test fun testEmptyList() = runTest { assertTrue(emptyList<Nothing>().asReceiveChannel().toList().isEmpty()) } diff --git a/kotlinx-coroutines-core/common/test/channels/ConflatedChannelArrayModelTest.kt b/kotlinx-coroutines-core/common/test/channels/ConflatedChannelArrayModelTest.kt deleted file mode 100644 index e80309be..00000000 --- a/kotlinx-coroutines-core/common/test/channels/ConflatedChannelArrayModelTest.kt +++ /dev/null @@ -1,11 +0,0 @@ -/* - * Copyright 2016-2020 JetBrains s.r.o. Use of this source code is governed by the Apache 2.0 license. - */ - -package kotlinx.coroutines.channels - -// Test that ArrayChannel(1, DROP_OLDEST) works just like ConflatedChannel() -class ConflatedChannelArrayModelTest : ConflatedChannelTest() { - override fun <T> createConflatedChannel(): Channel<T> = - ArrayChannel<T>(1, BufferOverflow.DROP_OLDEST, null) -} diff --git a/kotlinx-coroutines-core/common/test/channels/SendReceiveStressTest.kt b/kotlinx-coroutines-core/common/test/channels/SendReceiveStressTest.kt index b9aa9990..dcbb2d2f 100644 --- a/kotlinx-coroutines-core/common/test/channels/SendReceiveStressTest.kt +++ b/kotlinx-coroutines-core/common/test/channels/SendReceiveStressTest.kt @@ -5,7 +5,6 @@ package kotlinx.coroutines.channels import kotlinx.coroutines.* -import kotlin.coroutines.* import kotlin.test.* class SendReceiveStressTest : TestBase() { @@ -13,12 +12,12 @@ class SendReceiveStressTest : TestBase() { // Emulate parametrized by hand :( @Test - fun testArrayChannel() = runTest { + fun testBufferedChannel() = runTest { testStress(Channel(2)) } @Test - fun testLinkedListChannel() = runTest { + fun testUnlimitedChannel() = runTest { testStress(Channel(Channel.UNLIMITED)) } diff --git a/kotlinx-coroutines-core/common/test/channels/TestBroadcastChannelKind.kt b/kotlinx-coroutines-core/common/test/channels/TestBroadcastChannelKind.kt index d58c05da..94a48876 100644 --- a/kotlinx-coroutines-core/common/test/channels/TestBroadcastChannelKind.kt +++ b/kotlinx-coroutines-core/common/test/channels/TestBroadcastChannelKind.kt @@ -7,11 +7,11 @@ package kotlinx.coroutines.channels enum class TestBroadcastChannelKind { ARRAY_1 { override fun <T> create(): BroadcastChannel<T> = BroadcastChannel(1) - override fun toString(): String = "ArrayBroadcastChannel(1)" + override fun toString(): String = "BufferedBroadcastChannel(1)" }, ARRAY_10 { override fun <T> create(): BroadcastChannel<T> = BroadcastChannel(10) - override fun toString(): String = "ArrayBroadcastChannel(10)" + override fun toString(): String = "BufferedBroadcastChannel(10)" }, CONFLATED { override fun <T> create(): BroadcastChannel<T> = ConflatedBroadcastChannel() diff --git a/kotlinx-coroutines-core/common/test/channels/TestChannelKind.kt b/kotlinx-coroutines-core/common/test/channels/TestChannelKind.kt index f234e141..305c0eea 100644 --- a/kotlinx-coroutines-core/common/test/channels/TestChannelKind.kt +++ b/kotlinx-coroutines-core/common/test/channels/TestChannelKind.kt @@ -13,13 +13,13 @@ enum class TestChannelKind( val viaBroadcast: Boolean = false ) { RENDEZVOUS(0, "RendezvousChannel"), - ARRAY_1(1, "ArrayChannel(1)"), - ARRAY_2(2, "ArrayChannel(2)"), - ARRAY_10(10, "ArrayChannel(10)"), - LINKED_LIST(Channel.UNLIMITED, "LinkedListChannel"), + BUFFERED_1(1, "BufferedChannel(1)"), + BUFFERED_2(2, "BufferedChannel(2)"), + BUFFERED_10(10, "BufferedChannel(10)"), + UNLIMITED(Channel.UNLIMITED, "UnlimitedChannel"), CONFLATED(Channel.CONFLATED, "ConflatedChannel"), - ARRAY_1_BROADCAST(1, "ArrayBroadcastChannel(1)", viaBroadcast = true), - ARRAY_10_BROADCAST(10, "ArrayBroadcastChannel(10)", viaBroadcast = true), + BUFFERED_1_BROADCAST(1, "BufferedBroadcastChannel(1)", viaBroadcast = true), + BUFFERED_10_BROADCAST(10, "BufferedBroadcastChannel(10)", viaBroadcast = true), CONFLATED_BROADCAST(Channel.CONFLATED, "ConflatedBroadcastChannel", viaBroadcast = true) ; @@ -33,7 +33,7 @@ enum class TestChannelKind( override fun toString(): String = description } -private class ChannelViaBroadcast<E>( +internal class ChannelViaBroadcast<E>( private val broadcast: BroadcastChannel<E> ): Channel<E>, SendChannel<E> by broadcast { val sub = broadcast.openSubscription() @@ -46,11 +46,11 @@ private class ChannelViaBroadcast<E>( override fun iterator(): ChannelIterator<E> = sub.iterator() override fun tryReceive(): ChannelResult<E> = sub.tryReceive() - override fun cancel(cause: CancellationException?) = sub.cancel(cause) + override fun cancel(cause: CancellationException?) = broadcast.cancel(cause) // implementing hidden method anyway, so can cast to an internal class @Deprecated(level = DeprecationLevel.HIDDEN, message = "Since 1.2.0, binary compatibility with versions <= 1.1.x") - override fun cancel(cause: Throwable?): Boolean = (sub as AbstractChannel).cancelInternal(cause) + override fun cancel(cause: Throwable?): Boolean = error("unsupported") override val onReceive: SelectClause1<E> get() = sub.onReceive diff --git a/kotlinx-coroutines-core/common/test/channels/LinkedListChannelTest.kt b/kotlinx-coroutines-core/common/test/channels/UnlimitedChannelTest.kt index 501affb4..24b9d3d0 100644 --- a/kotlinx-coroutines-core/common/test/channels/LinkedListChannelTest.kt +++ b/kotlinx-coroutines-core/common/test/channels/UnlimitedChannelTest.kt @@ -7,7 +7,7 @@ package kotlinx.coroutines.channels import kotlinx.coroutines.* import kotlin.test.* -class LinkedListChannelTest : TestBase() { +class UnlimitedChannelTest : TestBase() { @Test fun testBasic() = runTest { val c = Channel<Int>(Channel.UNLIMITED) diff --git a/kotlinx-coroutines-core/common/test/flow/NamedDispatchers.kt b/kotlinx-coroutines-core/common/test/flow/NamedDispatchers.kt index 67bcbdc2..2459255c 100644 --- a/kotlinx-coroutines-core/common/test/flow/NamedDispatchers.kt +++ b/kotlinx-coroutines-core/common/test/flow/NamedDispatchers.kt @@ -5,12 +5,10 @@ package kotlinx.coroutines import kotlin.coroutines.* -import kotlin.native.concurrent.* /** * Test dispatchers that emulate multiplatform context tracking. */ -@ThreadLocal public object NamedDispatchers { private val stack = ArrayStack() diff --git a/kotlinx-coroutines-core/common/test/flow/operators/FilterTrivialTest.kt b/kotlinx-coroutines-core/common/test/flow/operators/FilterTrivialTest.kt index 1d3c69bc..f41fe731 100644 --- a/kotlinx-coroutines-core/common/test/flow/operators/FilterTrivialTest.kt +++ b/kotlinx-coroutines-core/common/test/flow/operators/FilterTrivialTest.kt @@ -29,6 +29,33 @@ class FilterTrivialTest : TestBase() { } @Test + fun testParametrizedFilterIsInstance() = runTest { + val flow = flowOf("value", 2.0) + assertEquals(2.0, flow.filterIsInstance(Double::class).single()) + assertEquals("value", flow.filterIsInstance(String::class).single()) + } + + @Test + fun testSubtypesFilterIsInstance() = runTest { + open class Super + class Sub : Super() + + val flow = flowOf(Super(), Super(), Super(), Sub(), Sub(), Sub()) + assertEquals(6, flow.filterIsInstance<Super>().count()) + assertEquals(3, flow.filterIsInstance<Sub>().count()) + } + + @Test + fun testSubtypesParametrizedFilterIsInstance() = runTest { + open class Super + class Sub : Super() + + val flow = flowOf(Super(), Super(), Super(), Sub(), Sub(), Sub()) + assertEquals(6, flow.filterIsInstance(Super::class).count()) + assertEquals(3, flow.filterIsInstance(Sub::class).count()) + } + + @Test fun testFilterIsInstanceNullable() = runTest { val flow = flowOf(1, 2, null) assertEquals(2, flow.filterIsInstance<Int>().count()) @@ -40,4 +67,10 @@ class FilterTrivialTest : TestBase() { val sum = emptyFlow<Int>().filterIsInstance<Int>().sum() assertEquals(0, sum) } + + @Test + fun testEmptyFlowParametrizedIsInstance() = runTest { + val sum = emptyFlow<Int>().filterIsInstance(Int::class).sum() + assertEquals(0, sum) + } } diff --git a/kotlinx-coroutines-core/common/test/flow/operators/LintTest.kt b/kotlinx-coroutines-core/common/test/flow/operators/LintTest.kt new file mode 100644 index 00000000..9cf6cbb9 --- /dev/null +++ b/kotlinx-coroutines-core/common/test/flow/operators/LintTest.kt @@ -0,0 +1,31 @@ +/* + * Copyright 2016-2023 JetBrains s.r.o. Use of this source code is governed by the Apache 2.0 license. + */ + +package kotlinx.coroutines.flow.operators + +import kotlinx.coroutines.* +import kotlinx.coroutines.flow.* +import kotlin.test.* + +class LintTest: TestBase() { + /** + * Tests that using [SharedFlow.toList] and similar functions by passing a mutable collection does add values + * to the provided collection. + */ + @Test + fun testSharedFlowToCollection() = runTest { + val sharedFlow = MutableSharedFlow<Int>() + val list = mutableListOf<Int>() + val set = mutableSetOf<Int>() + val jobs = listOf(suspend { sharedFlow.toList(list) }, { sharedFlow.toSet(set) }).map { + launch(Dispatchers.Unconfined) { it() } + } + repeat(10) { + sharedFlow.emit(it) + } + jobs.forEach { it.cancelAndJoin() } + assertEquals((0..9).toList(), list) + assertEquals((0..9).toSet(), set) + } +} diff --git a/kotlinx-coroutines-core/common/test/flow/operators/TimeoutTest.kt b/kotlinx-coroutines-core/common/test/flow/operators/TimeoutTest.kt new file mode 100644 index 00000000..c09882f2 --- /dev/null +++ b/kotlinx-coroutines-core/common/test/flow/operators/TimeoutTest.kt @@ -0,0 +1,231 @@ +/* + * Copyright 2016-2021 JetBrains s.r.o. Use of this source code is governed by the Apache 2.0 license. + */ + +package kotlinx.coroutines.flow.operators + +import kotlinx.coroutines.* +import kotlinx.coroutines.flow.* +import kotlin.test.* +import kotlin.time.Duration.Companion.milliseconds + +class TimeoutTest : TestBase() { + @Test + fun testBasic() = withVirtualTime { + expect(1) + val flow = flow { + expect(3) + emit("A") + delay(100) + emit("B") + delay(100) + emit("C") + expect(4) + delay(400) + expectUnreached() + } + + expect(2) + val list = mutableListOf<String>() + assertFailsWith<TimeoutCancellationException>(flow.timeout(300.milliseconds).onEach { list.add(it) }) + assertEquals(listOf("A", "B", "C"), list) + finish(5) + } + + @Test + fun testSingleNull() = withVirtualTime { + val flow = flow<Int?> { + emit(null) + delay(1) + expect(1) + }.timeout(2.milliseconds) + assertNull(flow.single()) + finish(2) + } + + @Test + fun testBasicCustomAction() = withVirtualTime { + expect(1) + val flow = flow { + expect(3) + emit("A") + delay(100) + emit("B") + delay(100) + emit("C") + expect(4) + delay(400) + expectUnreached() + } + + expect(2) + val list = mutableListOf<String>() + flow.timeout(300.milliseconds).catch { if (it is TimeoutCancellationException) emit("-1") }.collect { list.add(it) } + assertEquals(listOf("A", "B", "C", "-1"), list) + finish(5) + } + + @Test + fun testDelayedFirst() = withVirtualTime { + expect(1) + val flow = flow { + expect(3) + delay(100) + emit(1) + expect(4) + }.timeout(250.milliseconds) + expect(2) + assertEquals(1, flow.singleOrNull()) + finish(5) + } + + @Test + fun testEmpty() = withVirtualTime { + val flow = emptyFlow<Any?>().timeout(1.milliseconds) + assertNull(flow.singleOrNull()) + finish(1) + } + + @Test + fun testScalar() = withVirtualTime { + val flow = flowOf(1, 2, 3).timeout(1.milliseconds) + assertEquals(listOf(1, 2, 3), flow.toList()) + finish(1) + } + + @Test + fun testUpstreamError() = testUpstreamError(TestException()) + + @Test + fun testUpstreamErrorTimeoutException() = + testUpstreamError(TimeoutCancellationException("Timed out waiting for ${0} ms", Job())) + + @Test + fun testUpstreamErrorCancellationException() = testUpstreamError(CancellationException("")) + + private inline fun <reified T: Throwable> testUpstreamError(cause: T) = runTest { + try { + // Workaround for JS legacy bug + flow { + emit(1) + throw cause + }.timeout(1000.milliseconds).collect() + expectUnreached() + } catch (e: Throwable) { + assertTrue { e is T } + finish(1) + } + } + + @Test + fun testUpstreamExceptionsTakingPriority() = withVirtualTime { + val flow = flow<Unit> { + expect(2) + withContext(NonCancellable) { + delay(2.milliseconds) + } + assertFalse(currentCoroutineContext().isActive) // cancelled already + expect(3) + throw TestException() + }.timeout(1.milliseconds) + expect(1) + assertFailsWith<TestException> { + flow.collect { + expectUnreached() + } + } + finish(4) + } + + @Test + fun testDownstreamError() = runTest { + val flow = flow { + expect(1) + emit(1) + hang { expect(3) } + expectUnreached() + }.timeout(100.milliseconds).map { + expect(2) + yield() + throw TestException() + } + + assertFailsWith<TestException>(flow) + finish(4) + } + + @Test + fun testUpstreamTimeoutIsolatedContext() = withVirtualTime { + val flow = flow { + assertEquals("upstream", NamedDispatchers.name()) + expect(1) + emit(1) + expect(2) + delay(300) + expectUnreached() + }.flowOn(NamedDispatchers("upstream")).timeout(100.milliseconds) + + assertFailsWith<TimeoutCancellationException>(flow) + finish(3) + } + + @Test + fun testUpstreamTimeoutActionIsolatedContext() = withVirtualTime { + val flow = flow { + assertEquals("upstream", NamedDispatchers.name()) + expect(1) + emit(1) + expect(2) + delay(300) + expectUnreached() + }.flowOn(NamedDispatchers("upstream")).timeout(100.milliseconds).catch { + expect(3) + emit(2) + } + + assertEquals(listOf(1, 2), flow.toList()) + finish(4) + } + + @Test + fun testSharedFlowTimeout() = withVirtualTime { + // Workaround for JS legacy bug + try { + MutableSharedFlow<Int>().asSharedFlow().timeout(100.milliseconds).collect() + expectUnreached() + } catch (e: TimeoutCancellationException) { + finish(1) + } + } + + @Test + fun testSharedFlowCancelledNoTimeout() = runTest { + val mutableSharedFlow = MutableSharedFlow<Int>() + val list = arrayListOf<Int>() + + expect(1) + val consumerJob = launch { + expect(3) + mutableSharedFlow.asSharedFlow().timeout(100.milliseconds).collect { list.add(it) } + expectUnreached() + } + val producerJob = launch { + expect(4) + repeat(10) { + delay(50) + mutableSharedFlow.emit(it) + } + yield() + consumerJob.cancel() + expect(5) + } + + expect(2) + + producerJob.join() + consumerJob.join() + + assertEquals((0 until 10).toList(), list) + finish(6) + } +} diff --git a/kotlinx-coroutines-core/common/test/selects/SelectArrayChannelTest.kt b/kotlinx-coroutines-core/common/test/selects/SelectBufferedChannelTest.kt index 0158c843..6bb8049e 100644 --- a/kotlinx-coroutines-core/common/test/selects/SelectArrayChannelTest.kt +++ b/kotlinx-coroutines-core/common/test/selects/SelectBufferedChannelTest.kt @@ -6,10 +6,9 @@ package kotlinx.coroutines.selects import kotlinx.coroutines.* import kotlinx.coroutines.channels.* -import kotlinx.coroutines.intrinsics.* import kotlin.test.* -class SelectArrayChannelTest : TestBase() { +class SelectBufferedChannelTest : TestBase() { @Test fun testSelectSendSuccess() = runTest { @@ -383,11 +382,7 @@ class SelectArrayChannelTest : TestBase() { } // only for debugging - internal fun <R> SelectBuilder<R>.default(block: suspend () -> R) { - this as SelectBuilderImpl // type assertion - if (!trySelect()) return - block.startCoroutineUnintercepted(this) - } + internal fun <R> SelectBuilder<R>.default(block: suspend () -> R) = onTimeout(0, block) @Test fun testSelectReceiveOrClosedForClosedChannel() = runTest { diff --git a/kotlinx-coroutines-core/common/test/selects/SelectOldTest.kt b/kotlinx-coroutines-core/common/test/selects/SelectOldTest.kt new file mode 100644 index 00000000..34694fde --- /dev/null +++ b/kotlinx-coroutines-core/common/test/selects/SelectOldTest.kt @@ -0,0 +1,152 @@ +/* + * Copyright 2016-2021 JetBrains s.r.o. Use of this source code is governed by the Apache 2.0 license. + */ + +package kotlinx.coroutines.selects + +import kotlinx.coroutines.* +import kotlin.test.* + +class SelectOldTest : TestBase() { + @Test + fun testSelectCompleted() = runTest { + expect(1) + launch { // makes sure we don't yield to it earlier + finish(4) // after main exits + } + val job = Job() + job.cancel() + selectOld<Unit> { + job.onJoin { + expect(2) + } + } + expect(3) + // will wait for the first coroutine + } + + @Test + fun testSelectUnbiasedCompleted() = runTest { + expect(1) + launch { // makes sure we don't yield to it earlier + finish(4) // after main exits + } + val job = Job() + job.cancel() + selectUnbiasedOld<Unit> { + job.onJoin { + expect(2) + } + } + expect(3) + // will wait for the first coroutine + } + + @Test + fun testSelectIncomplete() = runTest { + expect(1) + val job = Job() + launch { // makes sure we don't yield to it earlier + expect(3) + val res = selectOld<String> { + job.onJoin { + expect(6) + "OK" + } + } + expect(7) + assertEquals("OK", res) + } + expect(2) + yield() + expect(4) + job.cancel() + expect(5) + yield() + finish(8) + } + + @Test + fun testSelectUnbiasedIncomplete() = runTest { + expect(1) + val job = Job() + launch { // makes sure we don't yield to it earlier + expect(3) + val res = selectUnbiasedOld<String> { + job.onJoin { + expect(6) + "OK" + } + } + expect(7) + assertEquals("OK", res) + } + expect(2) + yield() + expect(4) + job.cancel() + expect(5) + yield() + finish(8) + } + + @Test + fun testSelectUnbiasedComplete() = runTest { + expect(1) + val job = Job() + job.complete() + expect(2) + val res = selectUnbiasedOld<String> { + job.onJoin { + expect(3) + "OK" + } + } + assertEquals("OK", res) + finish(4) + } + + @Test + fun testSelectUnbiasedThrows() = runTest { + try { + select<Unit> { + expect(1) + throw TestException() + } + } catch (e: TestException) { + finish(2) + } + } + + @Test + fun testSelectLazy() = runTest { + expect(1) + val job = launch(start = CoroutineStart.LAZY) { + expect(2) + } + val res = selectOld<String> { + job.onJoin { + expect(3) + "OK" + } + } + finish(4) + assertEquals("OK", res) + } + + @Test + fun testSelectUnbiasedLazy() = runTest { + expect(1) + val job = launch(start = CoroutineStart.LAZY) { + expect(2) + } + val res = selectUnbiasedOld<String> { + job.onJoin { + expect(3) + "OK" + } + } + finish(4) + assertEquals("OK", res) + } +} diff --git a/kotlinx-coroutines-core/common/test/selects/SelectRendezvousChannelTest.kt b/kotlinx-coroutines-core/common/test/selects/SelectRendezvousChannelTest.kt index 6a157676..f3c5b4f3 100644 --- a/kotlinx-coroutines-core/common/test/selects/SelectRendezvousChannelTest.kt +++ b/kotlinx-coroutines-core/common/test/selects/SelectRendezvousChannelTest.kt @@ -7,7 +7,6 @@ package kotlinx.coroutines.selects import kotlinx.coroutines.* import kotlinx.coroutines.channels.* -import kotlinx.coroutines.intrinsics.* import kotlin.test.* class SelectRendezvousChannelTest : TestBase() { @@ -442,11 +441,7 @@ class SelectRendezvousChannelTest : TestBase() { } // only for debugging - internal fun <R> SelectBuilder<R>.default(block: suspend () -> R) { - this as SelectBuilderImpl // type assertion - if (!trySelect()) return - block.startCoroutineUnintercepted(this) - } + internal fun <R> SelectBuilder<R>.default(block: suspend () -> R) = onTimeout(0, block) @Test fun testSelectSendAndReceive() = runTest { diff --git a/kotlinx-coroutines-core/common/test/selects/SelectLinkedListChannelTest.kt b/kotlinx-coroutines-core/common/test/selects/SelectUnlimitedChannelTest.kt index a066f6b3..081c9183 100644 --- a/kotlinx-coroutines-core/common/test/selects/SelectLinkedListChannelTest.kt +++ b/kotlinx-coroutines-core/common/test/selects/SelectUnlimitedChannelTest.kt @@ -8,7 +8,7 @@ import kotlinx.coroutines.* import kotlinx.coroutines.channels.* import kotlin.test.* -class SelectLinkedListChannelTest : TestBase() { +class SelectUnlimitedChannelTest : TestBase() { @Test fun testSelectSendWhenClosed() = runTest { expect(1) diff --git a/kotlinx-coroutines-core/common/test/sync/MutexTest.kt b/kotlinx-coroutines-core/common/test/sync/MutexTest.kt index 4f428bc4..b4acd94e 100644 --- a/kotlinx-coroutines-core/common/test/sync/MutexTest.kt +++ b/kotlinx-coroutines-core/common/test/sync/MutexTest.kt @@ -4,8 +4,8 @@ package kotlinx.coroutines.sync -import kotlinx.atomicfu.* import kotlinx.coroutines.* +import kotlinx.coroutines.selects.* import kotlin.test.* class MutexTest : TestBase() { @@ -107,4 +107,45 @@ class MutexTest : TestBase() { assertFalse(mutex.holdsLock(firstOwner)) assertFalse(mutex.holdsLock(secondOwner)) } + + @Test + fun testUnlockWithNullOwner() { + val owner = Any() + val mutex = Mutex() + assertTrue(mutex.tryLock(owner)) + assertFailsWith<IllegalStateException> { mutex.unlock(Any()) } + mutex.unlock(null) + assertFalse(mutex.holdsLock(owner)) + assertFalse(mutex.isLocked) + } + + @Test + fun testUnlockWithoutOwnerWithLockedQueue() = runTest { + val owner = Any() + val owner2 = Any() + val mutex = Mutex() + assertTrue(mutex.tryLock(owner)) + expect(1) + launch { + expect(2) + mutex.lock(owner2) + } + yield() + expect(3) + assertFailsWith<IllegalStateException> { mutex.unlock(owner2) } + mutex.unlock() + assertFalse(mutex.holdsLock(owner)) + assertTrue(mutex.holdsLock(owner2)) + finish(4) + } + + @Test + fun testIllegalStateInvariant() = runTest { + val mutex = Mutex() + val owner = Any() + assertTrue(mutex.tryLock(owner)) + assertFailsWith<IllegalStateException> { mutex.tryLock(owner) } + assertFailsWith<IllegalStateException> { mutex.lock(owner) } + assertFailsWith<IllegalStateException> { select { mutex.onLock(owner) {} } } + } } diff --git a/kotlinx-coroutines-core/concurrent/src/Dispatchers.kt b/kotlinx-coroutines-core/concurrent/src/Dispatchers.kt new file mode 100644 index 00000000..8a937e38 --- /dev/null +++ b/kotlinx-coroutines-core/concurrent/src/Dispatchers.kt @@ -0,0 +1,37 @@ +/* + * Copyright 2016-2023 JetBrains s.r.o. Use of this source code is governed by the Apache 2.0 license. + */ + +package kotlinx.coroutines + +/** + * The [CoroutineDispatcher] that is designed for offloading blocking IO tasks to a shared pool of threads. + * Additional threads in this pool are created on demand. + * Default IO pool size is `64`; on JVM it can be configured using JVM-specific mechanisms, + * please refer to `Dispatchers.IO` documentation on JVM platform. + * + * ### Elasticity for limited parallelism + * + * `Dispatchers.IO` has a unique property of elasticity: its views + * obtained with [CoroutineDispatcher.limitedParallelism] are + * not restricted by the `Dispatchers.IO` parallelism. Conceptually, there is + * a dispatcher backed by an unlimited pool of threads, and both `Dispatchers.IO` + * and views of `Dispatchers.IO` are actually views of that dispatcher. In practice + * this means that, despite not abiding by `Dispatchers.IO`'s parallelism + * restrictions, its views share threads and resources with it. + * + * In the following example + * ``` + * // 100 threads for MySQL connection + * val myMysqlDbDispatcher = Dispatchers.IO.limitedParallelism(100) + * // 60 threads for MongoDB connection + * val myMongoDbDispatcher = Dispatchers.IO.limitedParallelism(60) + * ``` + * the system may have up to `64 + 100 + 60` threads dedicated to blocking tasks during peak loads, + * but during its steady state there is only a small number of threads shared + * among `Dispatchers.IO`, `myMysqlDbDispatcher` and `myMongoDbDispatcher` + */ +@Suppress("EXTENSION_SHADOWED_BY_MEMBER") +public expect val Dispatchers.IO: CoroutineDispatcher + + diff --git a/kotlinx-coroutines-core/concurrent/src/MultithreadedDispatchers.common.kt b/kotlinx-coroutines-core/concurrent/src/MultithreadedDispatchers.common.kt index a2b4241f..cc2c16a6 100644 --- a/kotlinx-coroutines-core/concurrent/src/MultithreadedDispatchers.common.kt +++ b/kotlinx-coroutines-core/concurrent/src/MultithreadedDispatchers.common.kt @@ -2,10 +2,40 @@ * Copyright 2016-2021 JetBrains s.r.o. Use of this source code is governed by the Apache 2.0 license. */ +@file:JvmMultifileClass +@file:JvmName("ThreadPoolDispatcherKt") package kotlinx.coroutines +import kotlin.jvm.* + +/** + * Creates a coroutine execution context using a single thread with built-in [yield] support. + * **NOTE: The resulting [CloseableCoroutineDispatcher] owns native resources (its thread). + * Resources are reclaimed by [CloseableCoroutineDispatcher.close].** + * + * If the resulting dispatcher is [closed][CloseableCoroutineDispatcher.close] and + * attempt to submit a task is made, then: + * * On the JVM, the [Job] of the affected task is [cancelled][Job.cancel] and the task is submitted to the + * [Dispatchers.IO], so that the affected coroutine can clean up its resources and promptly complete. + * * On Native, the attempt to submit a task throws an exception. + * + * This is a **delicate** API. The result of this method is a closeable resource with the + * associated native resources (threads or native workers). It should not be allocated in place, + * should be closed at the end of its lifecycle, and has non-trivial memory and CPU footprint. + * If you do not need a separate thread-pool, but only have to limit effective parallelism of the dispatcher, + * it is recommended to use [CoroutineDispatcher.limitedParallelism] instead. + * + * If you need a completely separate thread-pool with scheduling policy that is based on the standard + * JDK executors, use the following expression: + * `Executors.newSingleThreadExecutor().asCoroutineDispatcher()`. + * See `Executor.asCoroutineDispatcher` for details. + * + * @param name the base name of the created thread. + */ @ExperimentalCoroutinesApi -public expect fun newSingleThreadContext(name: String): CloseableCoroutineDispatcher +@DelicateCoroutinesApi +public fun newSingleThreadContext(name: String): CloseableCoroutineDispatcher = + newFixedThreadPoolContext(1, name) @ExperimentalCoroutinesApi public expect fun newFixedThreadPoolContext(nThreads: Int, name: String): CloseableCoroutineDispatcher diff --git a/kotlinx-coroutines-core/concurrent/src/channels/Channels.kt b/kotlinx-coroutines-core/concurrent/src/channels/Channels.kt index 24422b5a..c157677d 100644 --- a/kotlinx-coroutines-core/concurrent/src/channels/Channels.kt +++ b/kotlinx-coroutines-core/concurrent/src/channels/Channels.kt @@ -30,7 +30,7 @@ import kotlin.jvm.* public fun <E> SendChannel<E>.trySendBlocking(element: E): ChannelResult<Unit> { /* * Sent successfully -- bail out. - * But failure may indicate either that the channel it full or that + * But failure may indicate either that the channel is full or that * it is close. Go to slow path on failure to simplify the successful path and * to materialize default exception. */ @@ -44,11 +44,11 @@ public fun <E> SendChannel<E>.trySendBlocking(element: E): ChannelResult<Unit> { /** @suppress */ @Deprecated( - level = DeprecationLevel.ERROR, + level = DeprecationLevel.HIDDEN, message = "Deprecated in the favour of 'trySendBlocking'. " + "Consider handling the result of 'trySendBlocking' explicitly and rethrow exception if necessary", replaceWith = ReplaceWith("trySendBlocking(element)") -) // WARNING in 1.5.0, ERROR in 1.6.0, HIDDEN in 1.7.0 +) // WARNING in 1.5.0, ERROR in 1.6.0 public fun <E> SendChannel<E>.sendBlocking(element: E) { // fast path if (trySend(element).isSuccess) diff --git a/kotlinx-coroutines-core/concurrent/src/internal/LockFreeLinkedList.kt b/kotlinx-coroutines-core/concurrent/src/internal/LockFreeLinkedList.kt index b4b36dad..00888499 100644 --- a/kotlinx-coroutines-core/concurrent/src/internal/LockFreeLinkedList.kt +++ b/kotlinx-coroutines-core/concurrent/src/internal/LockFreeLinkedList.kt @@ -8,7 +8,6 @@ package kotlinx.coroutines.internal import kotlinx.atomicfu.* import kotlinx.coroutines.* import kotlin.jvm.* -import kotlin.native.concurrent.* private typealias Node = LockFreeLinkedListNode @@ -22,25 +21,8 @@ internal const val SUCCESS: Int = 1 internal const val FAILURE: Int = 2 @PublishedApi -@SharedImmutable internal val CONDITION_FALSE: Any = Symbol("CONDITION_FALSE") -@PublishedApi -@SharedImmutable -internal val LIST_EMPTY: Any = Symbol("LIST_EMPTY") - -/** @suppress **This is unstable API and it is subject to change.** */ -public actual typealias RemoveFirstDesc<T> = LockFreeLinkedListNode.RemoveFirstDesc<T> - -/** @suppress **This is unstable API and it is subject to change.** */ -public actual typealias AddLastDesc<T> = LockFreeLinkedListNode.AddLastDesc<T> - -/** @suppress **This is unstable API and it is subject to change.** */ -public actual typealias AbstractAtomicDesc = LockFreeLinkedListNode.AbstractAtomicDesc - -/** @suppress **This is unstable API and it is subject to change.** */ -public actual typealias PrepareOp = LockFreeLinkedListNode.PrepareOp - /** * Doubly-linked concurrent list node with remove support. * Based on paper @@ -145,8 +127,6 @@ public actual open class LockFreeLinkedListNode { } } - public fun <T : Node> describeAddLast(node: T): AddLastDesc<T> = AddLastDesc(this, node) - /** * Adds last item to this list atomically if the [condition] is true. */ @@ -161,30 +141,6 @@ public actual open class LockFreeLinkedListNode { } } - public actual inline fun addLastIfPrev(node: Node, predicate: (Node) -> Boolean): Boolean { - while (true) { // lock-free loop on prev.next - val prev = prevNode // sentinel node is never removed, so prev is always defined - if (!predicate(prev)) return false - if (prev.addNext(node, this)) return true - } - } - - public actual inline fun addLastIfPrevAndIf( - node: Node, - predicate: (Node) -> Boolean, // prev node predicate - crossinline condition: () -> Boolean // atomically checked condition - ): Boolean { - val condAdd = makeCondAddOp(node, condition) - while (true) { // lock-free loop on prev.next - val prev = prevNode // sentinel node is never removed, so prev is always defined - if (!predicate(prev)) return false - when (prev.tryCondAddNext(node, this, condAdd)) { - SUCCESS -> return true - FAILURE -> return false - } - } - } - // ------ addXXX util ------ /** @@ -239,7 +195,6 @@ public actual open class LockFreeLinkedListNode { * * **Note**: Invocation of this operation does not guarantee that remove was actually complete if result was `false`. * In particular, invoking [nextNode].[prevNode] might still return this node even though it is "already removed". - * Invoke [helpRemove] to make sure that remove was completed. */ public actual open fun remove(): Boolean = removeOrNext() == null @@ -260,263 +215,9 @@ public actual open class LockFreeLinkedListNode { } } - // Helps with removal of this node - public actual fun helpRemove() { - // Note: this node must be already removed - (next as Removed).ref.helpRemovePrev() - } - - // Helps with removal of nodes that are previous to this - @PublishedApi - internal fun helpRemovePrev() { - // We need to call correctPrev on a non-removed node to ensure progress, since correctPrev bails out when - // called on a removed node. There's always at least one non-removed node (list head). - var node = this - while (true) { - val next = node.next - if (next !is Removed) break - node = next.ref - } - // Found a non-removed node - node.correctPrev(null) - } - - public actual fun removeFirstOrNull(): Node? { - while (true) { // try to linearize - val first = next as Node - if (first === this) return null - if (first.remove()) return first - first.helpRemove() // must help remove to ensure lock-freedom - } - } - - public fun describeRemoveFirst(): RemoveFirstDesc<Node> = RemoveFirstDesc(this) - - // just peek at item when predicate is true - public actual inline fun <reified T> removeFirstIfIsInstanceOfOrPeekIf(predicate: (T) -> Boolean): T? { - while (true) { - val first = this.next as Node - if (first === this) return null // got list head -- nothing to remove - if (first !is T) return null - if (predicate(first)) { - // check for removal of the current node to make sure "peek" operation is linearizable - if (!first.isRemoved) return first - } - val next = first.removeOrNext() - if (next === null) return first // removed successfully -- return it - // help and start from the beginning - next.helpRemovePrev() - } - } - - // ------ multi-word atomic operations helpers ------ - - public open class AddLastDesc<T : Node> constructor( - @JvmField val queue: Node, - @JvmField val node: T - ) : AbstractAtomicDesc() { - init { - // require freshly allocated node here - assert { node._next.value === node && node._prev.value === node } - } - - // Returns null when atomic op got into deadlock trying to help operation that started later (RETRY_ATOMIC) - final override fun takeAffectedNode(op: OpDescriptor): Node? = - queue.correctPrev(op) // queue head is never removed, so null result can only mean RETRY_ATOMIC - - private val _affectedNode = atomic<Node?>(null) - final override val affectedNode: Node? get() = _affectedNode.value - final override val originalNext: Node get() = queue - - override fun retry(affected: Node, next: Any): Boolean = next !== queue - - override fun finishPrepare(prepareOp: PrepareOp) { - // Note: onPrepare must use CAS to make sure the stale invocation is not - // going to overwrite the previous decision on successful preparation. - // Result of CAS is irrelevant, but we must ensure that it is set when invoker completes - _affectedNode.compareAndSet(null, prepareOp.affected) - } - - override fun updatedNext(affected: Node, next: Node): Any { - // it is invoked only on successfully completion of operation, but this invocation can be stale, - // so we must use CAS to set both prev & next pointers - node._prev.compareAndSet(node, affected) - node._next.compareAndSet(node, queue) - return node - } - - override fun finishOnSuccess(affected: Node, next: Node) { - node.finishAdd(queue) - } - } - - public open class RemoveFirstDesc<T>( - @JvmField val queue: Node - ) : AbstractAtomicDesc() { - private val _affectedNode = atomic<Node?>(null) - private val _originalNext = atomic<Node?>(null) - - @Suppress("UNCHECKED_CAST") - public val result: T get() = affectedNode!! as T - - final override fun takeAffectedNode(op: OpDescriptor): Node? { - queue._next.loop { next -> - if (next is OpDescriptor) { - if (op.isEarlierThan(next)) - return null // RETRY_ATOMIC - next.perform(queue) - } else { - return next as Node - } - } - } - - final override val affectedNode: Node? get() = _affectedNode.value - final override val originalNext: Node? get() = _originalNext.value - - // check node predicates here, must signal failure if affect is not of type T - protected override fun failure(affected: Node): Any? = - if (affected === queue) LIST_EMPTY else null - - final override fun retry(affected: Node, next: Any): Boolean { - if (next !is Removed) return false - next.ref.helpRemovePrev() // must help delete to ensure lock-freedom - return true - } - - override fun finishPrepare(prepareOp: PrepareOp) { - // Note: finishPrepare must use CAS to make sure the stale invocation is not - // going to overwrite the previous decision on successful preparation. - // Result of CAS is irrelevant, but we must ensure that it is set when invoker completes - _affectedNode.compareAndSet(null, prepareOp.affected) - _originalNext.compareAndSet(null, prepareOp.next) - } - - final override fun updatedNext(affected: Node, next: Node): Any = next.removed() - - final override fun finishOnSuccess(affected: Node, next: Node) { - // Complete removal operation here. It bails out if next node is also removed. It becomes - // responsibility of the next's removes to call correctPrev which would help fix all the links. - next.correctPrev(null) - } - } - // This is Harris's RDCSS (Restricted Double-Compare Single Swap) operation // It inserts "op" descriptor of when "op" status is still undecided (rolls back otherwise) - public class PrepareOp( - @JvmField val affected: Node, - @JvmField val next: Node, - @JvmField val desc: AbstractAtomicDesc - ) : OpDescriptor() { - override val atomicOp: AtomicOp<*> get() = desc.atomicOp - - // Returns REMOVE_PREPARED or null (it makes decision on any failure) - override fun perform(affected: Any?): Any? { - assert { affected === this.affected } - affected as Node // type assertion - val decision = desc.onPrepare(this) - if (decision === REMOVE_PREPARED) { - // remove element on failure -- do not mark as decided, will try another one - val next = this.next - val removed = next.removed() - if (affected._next.compareAndSet(this, removed)) { - // The element was actually removed - desc.onRemoved(affected) - // Complete removal operation here. It bails out if next node is also removed and it becomes - // responsibility of the next's removes to call correctPrev which would help fix all the links. - next.correctPrev(null) - } - return REMOVE_PREPARED - } - // We need to ensure progress even if it operation result consensus was already decided - val consensus = if (decision != null) { - // some other logic failure, including RETRY_ATOMIC -- reach consensus on decision fail reason ASAP - atomicOp.decide(decision) - } else { - atomicOp.consensus // consult with current decision status like in Harris DCSS - } - val update: Any = when { - consensus === NO_DECISION -> atomicOp // desc.onPrepare returned null -> start doing atomic op - consensus == null -> desc.updatedNext(affected, next) // move forward if consensus on success - else -> next // roll back if consensus if failure - } - affected._next.compareAndSet(this, update) - return null - } - - public fun finishPrepare(): Unit = desc.finishPrepare(this) - - override fun toString(): String = "PrepareOp(op=$atomicOp)" - } - - public abstract class AbstractAtomicDesc : AtomicDesc() { - protected abstract val affectedNode: Node? - protected abstract val originalNext: Node? - protected open fun takeAffectedNode(op: OpDescriptor): Node? = affectedNode!! // null for RETRY_ATOMIC - protected open fun failure(affected: Node): Any? = null // next: Node | Removed - protected open fun retry(affected: Node, next: Any): Boolean = false // next: Node | Removed - protected abstract fun finishOnSuccess(affected: Node, next: Node) - - public abstract fun updatedNext(affected: Node, next: Node): Any - - public abstract fun finishPrepare(prepareOp: PrepareOp) - - // non-null on failure - public open fun onPrepare(prepareOp: PrepareOp): Any? { - finishPrepare(prepareOp) - return null - } - public open fun onRemoved(affected: Node) {} // called once when node was prepared & later removed - - @Suppress("UNCHECKED_CAST") - final override fun prepare(op: AtomicOp<*>): Any? { - while (true) { // lock free loop on next - val affected = takeAffectedNode(op) ?: return RETRY_ATOMIC - // read its original next pointer first - val next = affected._next.value - // then see if already reached consensus on overall operation - if (next === op) return null // already in process of operation -- all is good - if (op.isDecided) return null // already decided this operation -- go to next desc - if (next is OpDescriptor) { - // some other operation is in process - // if operation in progress (preparing or prepared) has higher sequence number -- abort our preparations - if (op.isEarlierThan(next)) - return RETRY_ATOMIC - next.perform(affected) - continue // and retry - } - // next: Node | Removed - val failure = failure(affected) - if (failure != null) return failure // signal failure - if (retry(affected, next)) continue // retry operation - val prepareOp = PrepareOp(affected, next as Node, this) - if (affected._next.compareAndSet(next, prepareOp)) { - // prepared -- complete preparations - try { - val prepFail = prepareOp.perform(affected) - if (prepFail === REMOVE_PREPARED) continue // retry - assert { prepFail == null } - return null - } catch (e: Throwable) { - // Crashed during preparation (for example IllegalStateExpception) -- undo & rethrow - affected._next.compareAndSet(prepareOp, next) - throw e - } - } - } - } - - final override fun complete(op: AtomicOp<*>, failure: Any?) { - val success = failure == null - val affectedNode = affectedNode ?: run { assert { !success }; return } - val originalNext = originalNext ?: run { assert { !success }; return } - val update = if (success) updatedNext(affectedNode, originalNext) else originalNext - if (affectedNode._next.compareAndSet(op, update)) { - if (success) finishOnSuccess(affectedNode, originalNext) - } - } - } // ------ other helpers ------ @@ -565,9 +266,6 @@ public actual open class LockFreeLinkedListNode { * * When this node is removed. In this case there is no need to waste time on corrections, because * remover of this node will ultimately call [correctPrev] on the next node and that will fix all * the links from this node, too. - * * When [op] descriptor is not `null` and operation descriptor that is [OpDescriptor.isEarlierThan] - * that current [op] is found while traversing the list. This `null` result will be translated - * by callers to [RETRY_ATOMIC]. */ private tailrec fun correctPrev(op: OpDescriptor?): Node? { val oldPrev = _prev.value @@ -590,8 +288,6 @@ public actual open class LockFreeLinkedListNode { this.isRemoved -> return null // nothing to do, this node was removed, bail out asap to save time prevNext === op -> return prev // part of the same op -- don't recurse, didn't correct prev prevNext is OpDescriptor -> { // help & retry - if (op != null && op.isEarlierThan(prevNext)) - return null // RETRY_ATOMIC prevNext.perform(prev) return correctPrev(op) // retry from scratch } diff --git a/kotlinx-coroutines-core/concurrent/src/internal/OnDemandAllocatingPool.kt b/kotlinx-coroutines-core/concurrent/src/internal/OnDemandAllocatingPool.kt new file mode 100644 index 00000000..1c2beff2 --- /dev/null +++ b/kotlinx-coroutines-core/concurrent/src/internal/OnDemandAllocatingPool.kt @@ -0,0 +1,106 @@ +/* + * Copyright 2016-2022 JetBrains s.r.o. Use of this source code is governed by the Apache 2.0 license. + */ + +package kotlinx.coroutines.internal + +import kotlinx.atomicfu.* + +/** + * A thread-safe resource pool. + * + * [maxCapacity] is the maximum amount of elements. + * [create] is the function that creates a new element. + * + * This is only used in the Native implementation, + * but is part of the `concurrent` source set in order to test it on the JVM. + */ +internal class OnDemandAllocatingPool<T>( + private val maxCapacity: Int, + private val create: (Int) -> T +) { + /** + * Number of existing elements + isClosed flag in the highest bit. + * Once the flag is set, the value is guaranteed not to change anymore. + */ + private val controlState = atomic(0) + private val elements = atomicArrayOfNulls<T>(maxCapacity) + + /** + * Returns the number of elements that need to be cleaned up due to the pool being closed. + */ + @Suppress("NOTHING_TO_INLINE") + private inline fun tryForbidNewElements(): Int { + controlState.loop { + if (it.isClosed()) return 0 // already closed + if (controlState.compareAndSet(it, it or IS_CLOSED_MASK)) return it + } + } + + @Suppress("NOTHING_TO_INLINE") + private inline fun Int.isClosed(): Boolean = this and IS_CLOSED_MASK != 0 + + /** + * Request that a new element is created. + * + * Returns `false` if the pool is closed. + * + * Note that it will still return `true` even if an element was not created due to reaching [maxCapacity]. + * + * Rethrows the exceptions thrown from [create]. In this case, this operation has no effect. + */ + fun allocate(): Boolean { + controlState.loop { ctl -> + if (ctl.isClosed()) return false + if (ctl >= maxCapacity) return true + if (controlState.compareAndSet(ctl, ctl + 1)) { + elements[ctl].value = create(ctl) + return true + } + } + } + + /** + * Close the pool. + * + * This will prevent any new elements from being created. + * All the elements present in the pool will be returned. + * + * The function is thread-safe. + * + * [close] can be called multiple times, but only a single call will return a non-empty list. + * This is due to the elements being cleaned out from the pool on the first invocation to avoid memory leaks, + * and no new elements being created after. + */ + fun close(): List<T> { + val elementsExisting = tryForbidNewElements() + return (0 until elementsExisting).map { i -> + // we wait for the element to be created, because we know that eventually it is going to be there + loop { + val element = elements[i].getAndSet(null) + if (element != null) { + return@map element + } + } + } + } + + // for tests + internal fun stateRepresentation(): String { + val ctl = controlState.value + val elementsStr = (0 until (ctl and IS_CLOSED_MASK.inv())).map { elements[it].value }.toString() + val closedStr = if (ctl.isClosed()) "[closed]" else "" + return elementsStr + closedStr + } + + override fun toString(): String = "OnDemandAllocatingPool(${stateRepresentation()})" +} + +// KT-25023 +private inline fun loop(block: () -> Unit): Nothing { + while (true) { + block() + } +} + +private const val IS_CLOSED_MASK = 1 shl 31 diff --git a/kotlinx-coroutines-core/concurrent/test/AbstractDispatcherConcurrencyTest.kt b/kotlinx-coroutines-core/concurrent/test/AbstractDispatcherConcurrencyTest.kt index 8fc4f4ef..7dc500f5 100644 --- a/kotlinx-coroutines-core/concurrent/test/AbstractDispatcherConcurrencyTest.kt +++ b/kotlinx-coroutines-core/concurrent/test/AbstractDispatcherConcurrencyTest.kt @@ -12,7 +12,7 @@ abstract class AbstractDispatcherConcurrencyTest : TestBase() { public abstract val dispatcher: CoroutineDispatcher @Test - fun testLaunchAndJoin() = runMtTest { + fun testLaunchAndJoin() = runTest { expect(1) var capturedMutableState = 0 val job = GlobalScope.launch(dispatcher) { @@ -25,7 +25,7 @@ abstract class AbstractDispatcherConcurrencyTest : TestBase() { } @Test - fun testDispatcherHasOwnThreads() = runMtTest { + fun testDispatcherHasOwnThreads() = runTest { val channel = Channel<Int>() GlobalScope.launch(dispatcher) { channel.send(42) @@ -41,7 +41,7 @@ abstract class AbstractDispatcherConcurrencyTest : TestBase() { } @Test - fun testDelayInDispatcher() = runMtTest { + fun testDelayInDispatcher() = runTest { expect(1) val job = GlobalScope.launch(dispatcher) { expect(2) diff --git a/kotlinx-coroutines-core/concurrent/test/AtomicCancellationTest.kt b/kotlinx-coroutines-core/concurrent/test/AtomicCancellationTest.kt index 74751fcc..7bbd7ebe 100644 --- a/kotlinx-coroutines-core/concurrent/test/AtomicCancellationTest.kt +++ b/kotlinx-coroutines-core/concurrent/test/AtomicCancellationTest.kt @@ -25,6 +25,7 @@ class AtomicCancellationTest : TestBase() { finish(4) } + @Suppress("UNUSED_VARIABLE") @Test fun testSelectSendCancellable() = runBlocking { expect(1) diff --git a/kotlinx-coroutines-core/concurrent/test/CommonThreadLocalTest.kt b/kotlinx-coroutines-core/concurrent/test/CommonThreadLocalTest.kt new file mode 100644 index 00000000..598a96cc --- /dev/null +++ b/kotlinx-coroutines-core/concurrent/test/CommonThreadLocalTest.kt @@ -0,0 +1,75 @@ +/* + * Copyright 2016-2022 JetBrains s.r.o. Use of this source code is governed by the Apache 2.0 license. + */ + +package kotlinx.coroutines + +import kotlinx.coroutines.exceptions.* +import kotlinx.coroutines.internal.* +import kotlin.test.* + +class CommonThreadLocalTest: TestBase() { + + /** + * Tests the basic functionality of [commonThreadLocal]: storing a separate value for each thread. + */ + @Test + fun testThreadLocalBeingThreadLocal() = runTest { + val threadLocal = commonThreadLocal<Int>(Symbol("Test1")) + newSingleThreadContext("").use { + threadLocal.set(10) + assertEquals(10, threadLocal.get()) + val job1 = launch(it) { + threadLocal.set(20) + assertEquals(20, threadLocal.get()) + } + assertEquals(10, threadLocal.get()) + job1.join() + val job2 = launch(it) { + assertEquals(20, threadLocal.get()) + } + job2.join() + } + } + + /** + * Tests using [commonThreadLocal] with a nullable type. + */ + @Test + fun testThreadLocalWithNullableType() = runTest { + val threadLocal = commonThreadLocal<Int?>(Symbol("Test2")) + newSingleThreadContext("").use { + assertNull(threadLocal.get()) + threadLocal.set(10) + assertEquals(10, threadLocal.get()) + val job1 = launch(it) { + assertNull(threadLocal.get()) + threadLocal.set(20) + assertEquals(20, threadLocal.get()) + } + assertEquals(10, threadLocal.get()) + job1.join() + threadLocal.set(null) + assertNull(threadLocal.get()) + val job2 = launch(it) { + assertEquals(20, threadLocal.get()) + threadLocal.set(null) + assertNull(threadLocal.get()) + } + job2.join() + } + } + + /** + * Tests that several instances of [commonThreadLocal] with different names don't affect each other. + */ + @Test + fun testThreadLocalsWithDifferentNamesNotInterfering() { + val value1 = commonThreadLocal<Int>(Symbol("Test3a")) + val value2 = commonThreadLocal<Int>(Symbol("Test3b")) + value1.set(5) + value2.set(6) + assertEquals(5, value1.get()) + assertEquals(6, value2.get()) + } +} diff --git a/kotlinx-coroutines-core/concurrent/test/ConcurrentExceptionsStressTest.kt b/kotlinx-coroutines-core/concurrent/test/ConcurrentExceptionsStressTest.kt index d4252da3..cbfa3323 100644 --- a/kotlinx-coroutines-core/concurrent/test/ConcurrentExceptionsStressTest.kt +++ b/kotlinx-coroutines-core/concurrent/test/ConcurrentExceptionsStressTest.kt @@ -22,7 +22,7 @@ class ConcurrentExceptionsStressTest : TestBase() { } @Test - fun testStress() = runMtTest { + fun testStress() = runTest { workers = Array(nWorkers) { index -> newSingleThreadContext("JobExceptionsStressTest-$index") } diff --git a/kotlinx-coroutines-core/concurrent/test/ConcurrentTestUtilities.common.kt b/kotlinx-coroutines-core/concurrent/test/ConcurrentTestUtilities.common.kt index a4d40fb2..d40f118d 100644 --- a/kotlinx-coroutines-core/concurrent/test/ConcurrentTestUtilities.common.kt +++ b/kotlinx-coroutines-core/concurrent/test/ConcurrentTestUtilities.common.kt @@ -8,6 +8,7 @@ import kotlinx.coroutines.* internal expect open class SuppressSupportingThrowable() : Throwable expect val Throwable.suppressed: Array<Throwable> +// Unused on purpose, used manually during debugging sessions expect fun Throwable.printStackTrace() expect fun randomWait() diff --git a/kotlinx-coroutines-core/concurrent/test/DefaultDispatcherConcurrencyTest.kt b/kotlinx-coroutines-core/concurrent/test/DefaultDispatchersConcurrencyTest.kt index a12930cc..66dfd07a 100644 --- a/kotlinx-coroutines-core/concurrent/test/DefaultDispatcherConcurrencyTest.kt +++ b/kotlinx-coroutines-core/concurrent/test/DefaultDispatchersConcurrencyTest.kt @@ -6,3 +6,7 @@ package kotlinx.coroutines class DefaultDispatcherConcurrencyTest : AbstractDispatcherConcurrencyTest() { override val dispatcher: CoroutineDispatcher = Dispatchers.Default } + +class IoDispatcherConcurrencyTest : AbstractDispatcherConcurrencyTest() { + override val dispatcher: CoroutineDispatcher = Dispatchers.IO +} diff --git a/kotlinx-coroutines-core/concurrent/test/JobStructuredJoinStressTest.kt b/kotlinx-coroutines-core/concurrent/test/JobStructuredJoinStressTest.kt index 431bb697..4b5c952d 100644 --- a/kotlinx-coroutines-core/concurrent/test/JobStructuredJoinStressTest.kt +++ b/kotlinx-coroutines-core/concurrent/test/JobStructuredJoinStressTest.kt @@ -16,12 +16,12 @@ class JobStructuredJoinStressTest : TestBase() { private val nRepeats = 10_000 * stressTestMultiplier @Test - fun testStressRegularJoin() = runMtTest { + fun testStressRegularJoin() = runTest { stress(Job::join) } @Test - fun testStressSuspendCancellable() = runMtTest { + fun testStressSuspendCancellable() = runTest { stress { job -> suspendCancellableCoroutine { cont -> job.invokeOnCompletion { cont.resume(Unit) } @@ -30,7 +30,7 @@ class JobStructuredJoinStressTest : TestBase() { } @Test - fun testStressSuspendCancellableReusable() = runMtTest { + fun testStressSuspendCancellableReusable() = runTest { stress { job -> suspendCancellableCoroutineReusable { cont -> job.invokeOnCompletion { cont.resume(Unit) } diff --git a/kotlinx-coroutines-core/concurrent/test/LimitedParallelismConcurrentTest.kt b/kotlinx-coroutines-core/concurrent/test/LimitedParallelismConcurrentTest.kt index 8d38f05b..49fe93f6 100644 --- a/kotlinx-coroutines-core/concurrent/test/LimitedParallelismConcurrentTest.kt +++ b/kotlinx-coroutines-core/concurrent/test/LimitedParallelismConcurrentTest.kt @@ -7,6 +7,7 @@ package kotlinx.coroutines import kotlinx.atomicfu.* import kotlinx.coroutines.* import kotlinx.coroutines.exceptions.* +import kotlin.coroutines.* import kotlin.test.* class LimitedParallelismConcurrentTest : TestBase() { @@ -23,7 +24,7 @@ class LimitedParallelismConcurrentTest : TestBase() { } @Test - fun testLimitedExecutor() = runMtTest { + fun testLimitedExecutor() = runTest { val executor = newFixedThreadPoolContext(targetParallelism, "test") val view = executor.limitedParallelism(targetParallelism) doStress { @@ -45,7 +46,7 @@ class LimitedParallelismConcurrentTest : TestBase() { } @Test - fun testTaskFairness() = runMtTest { + fun testTaskFairness() = runTest { val executor = newSingleThreadContext("test") val view = executor.limitedParallelism(1) val view2 = executor.limitedParallelism(1) @@ -58,4 +59,37 @@ class LimitedParallelismConcurrentTest : TestBase() { joinAll(j1, j2) executor.close() } + + /** + * Tests that, when no tasks are present, the limited dispatcher does not dispatch any tasks. + * This is important for the case when a dispatcher is closeable and the [CoroutineDispatcher.limitedParallelism] + * machinery could trigger a dispatch after the dispatcher is closed. + */ + @Test + fun testNotDoingDispatchesWhenNoTasksArePresent() = runTest { + class NaggingDispatcher: CoroutineDispatcher() { + val closed = atomic(false) + override fun dispatch(context: CoroutineContext, block: Runnable) { + if (closed.value) + fail("Dispatcher was closed, but still dispatched a task") + Dispatchers.Default.dispatch(context, block) + } + fun close() { + closed.value = true + } + } + repeat(stressTestMultiplier * 500_000) { + val dispatcher = NaggingDispatcher() + val view = dispatcher.limitedParallelism(1) + val deferred = CompletableDeferred<Unit>() + val job = launch(view) { + deferred.await() + } + launch(Dispatchers.Default) { + deferred.complete(Unit) + } + job.join() + dispatcher.close() + } + } } diff --git a/kotlinx-coroutines-core/concurrent/test/MultithreadedDispatcherStressTest.kt b/kotlinx-coroutines-core/concurrent/test/MultithreadedDispatcherStressTest.kt new file mode 100644 index 00000000..4e4583f2 --- /dev/null +++ b/kotlinx-coroutines-core/concurrent/test/MultithreadedDispatcherStressTest.kt @@ -0,0 +1,35 @@ +/* + * Copyright 2016-2023 JetBrains s.r.o. Use of this source code is governed by the Apache 2.0 license. + */ + +package kotlinx.coroutines + +import kotlinx.atomicfu.* +import kotlin.coroutines.* +import kotlin.test.* + +class MultithreadedDispatcherStressTest { + val shared = atomic(0) + + /** + * Tests that [newFixedThreadPoolContext] will not drop tasks when closed. + */ + @Test + fun testClosingNotDroppingTasks() { + repeat(7) { + shared.value = 0 + val nThreads = it + 1 + val dispatcher = newFixedThreadPoolContext(nThreads, "testMultiThreadedContext") + repeat(1_000) { + dispatcher.dispatch(EmptyCoroutineContext, Runnable { + shared.incrementAndGet() + }) + } + dispatcher.close() + while (shared.value < 1_000) { + // spin. + // the test will hang here if the dispatcher drops tasks. + } + } + } +} diff --git a/kotlinx-coroutines-core/concurrent/test/RunBlockingTest.kt b/kotlinx-coroutines-core/concurrent/test/RunBlockingTest.kt index 70f6b8ba..f4844761 100644 --- a/kotlinx-coroutines-core/concurrent/test/RunBlockingTest.kt +++ b/kotlinx-coroutines-core/concurrent/test/RunBlockingTest.kt @@ -10,7 +10,7 @@ import kotlin.test.* class RunBlockingTest : TestBase() { @Test - fun testWithTimeoutBusyWait() = runMtTest { + fun testWithTimeoutBusyWait() = runTest { val value = withTimeoutOrNull(10) { while (isActive) { // Busy wait @@ -52,7 +52,7 @@ class RunBlockingTest : TestBase() { } @Test - fun testOtherDispatcher() = runMtTest { + fun testOtherDispatcher() = runTest { expect(1) val name = "RunBlockingTest.testOtherDispatcher" val thread = newSingleThreadContext(name) @@ -68,7 +68,7 @@ class RunBlockingTest : TestBase() { } @Test - fun testCancellation() = runMtTest { + fun testCancellation() = runTest { newFixedThreadPoolContext(2, "testCancellation").use { val job = GlobalScope.launch(it) { runBlocking(coroutineContext) { diff --git a/kotlinx-coroutines-core/concurrent/test/TestBaseExtension.common.kt b/kotlinx-coroutines-core/concurrent/test/TestBaseExtension.common.kt deleted file mode 100644 index b19bf50e..00000000 --- a/kotlinx-coroutines-core/concurrent/test/TestBaseExtension.common.kt +++ /dev/null @@ -1,10 +0,0 @@ -/* - * Copyright 2016-2021 JetBrains s.r.o. Use of this source code is governed by the Apache 2.0 license. - */ -package kotlinx.coroutines - -expect fun TestBase.runMtTest( - expected: ((Throwable) -> Boolean)? = null, - unhandled: List<(Throwable) -> Boolean> = emptyList(), - block: suspend CoroutineScope.() -> Unit -): TestResult diff --git a/kotlinx-coroutines-core/concurrent/test/channels/BroadcastChannelSubStressTest.kt b/kotlinx-coroutines-core/concurrent/test/channels/BroadcastChannelSubStressTest.kt index 30b1075c..245a80c2 100644 --- a/kotlinx-coroutines-core/concurrent/test/channels/BroadcastChannelSubStressTest.kt +++ b/kotlinx-coroutines-core/concurrent/test/channels/BroadcastChannelSubStressTest.kt @@ -20,7 +20,7 @@ class BroadcastChannelSubStressTest: TestBase() { private val receivedTotal = atomic(0L) @Test - fun testStress() = runMtTest { + fun testStress() = runTest { TestBroadcastChannelKind.values().forEach { kind -> println("--- BroadcastChannelSubStressTest $kind") val broadcast = kind.create<Long>() diff --git a/kotlinx-coroutines-core/concurrent/test/channels/ChannelCancelUndeliveredElementStressTest.kt b/kotlinx-coroutines-core/concurrent/test/channels/ChannelCancelUndeliveredElementStressTest.kt index 3e38eec3..1cf7d8a1 100644 --- a/kotlinx-coroutines-core/concurrent/test/channels/ChannelCancelUndeliveredElementStressTest.kt +++ b/kotlinx-coroutines-core/concurrent/test/channels/ChannelCancelUndeliveredElementStressTest.kt @@ -28,7 +28,7 @@ class ChannelCancelUndeliveredElementStressTest : TestBase() { private val dUndeliveredCnt = atomic(0) @Test - fun testStress() = runMtTest { + fun testStress() = runTest { repeat(repeatTimes) { val channel = Channel<Int>(1) { dUndeliveredCnt.incrementAndGet() } val j1 = launch(Dispatchers.Default) { @@ -51,6 +51,7 @@ class ChannelCancelUndeliveredElementStressTest : TestBase() { println(" Undelivered: ${dUndeliveredCnt.value}") error("Failed") } + (channel as? BufferedChannel<*>)?.checkSegmentStructureInvariants() trySendFailedCnt += dTrySendFailedCnt receivedCnt += dReceivedCnt undeliveredCnt += dUndeliveredCnt.value diff --git a/kotlinx-coroutines-core/concurrent/test/channels/ConflatedBroadcastChannelNotifyStressTest.kt b/kotlinx-coroutines-core/concurrent/test/channels/ConflatedBroadcastChannelNotifyStressTest.kt index 5da00d2a..d9ec7ad5 100644 --- a/kotlinx-coroutines-core/concurrent/test/channels/ConflatedBroadcastChannelNotifyStressTest.kt +++ b/kotlinx-coroutines-core/concurrent/test/channels/ConflatedBroadcastChannelNotifyStressTest.kt @@ -22,7 +22,7 @@ class ConflatedBroadcastChannelNotifyStressTest : TestBase() { private val receivedTotal = atomic(0) @Test - fun testStressNotify()= runMtTest { + fun testStressNotify()= runTest { println("--- ConflatedBroadcastChannelNotifyStressTest") val senders = List(nSenders) { senderId -> launch(Dispatchers.Default + CoroutineName("Sender$senderId")) { diff --git a/kotlinx-coroutines-core/concurrent/test/flow/CombineStressTest.kt b/kotlinx-coroutines-core/concurrent/test/flow/CombineStressTest.kt index f262e78f..ea6f96bb 100644 --- a/kotlinx-coroutines-core/concurrent/test/flow/CombineStressTest.kt +++ b/kotlinx-coroutines-core/concurrent/test/flow/CombineStressTest.kt @@ -10,7 +10,7 @@ import kotlin.test.* class CombineStressTest : TestBase() { @Test - fun testCancellation() = runMtTest { + fun testCancellation() = runTest { withContext(Dispatchers.Default + CoroutineExceptionHandler { _, _ -> expectUnreached() }) { flow { expect(1) @@ -26,7 +26,7 @@ class CombineStressTest : TestBase() { } @Test - fun testFailure() = runMtTest { + fun testFailure() = runTest { val innerIterations = 100 * stressTestMultiplierSqrt val outerIterations = 10 * stressTestMultiplierSqrt withContext(Dispatchers.Default + CoroutineExceptionHandler { _, _ -> expectUnreached() }) { diff --git a/kotlinx-coroutines-core/concurrent/test/flow/FlowCancellationTest.kt b/kotlinx-coroutines-core/concurrent/test/flow/FlowCancellationTest.kt index 286ba751..8680ff7d 100644 --- a/kotlinx-coroutines-core/concurrent/test/flow/FlowCancellationTest.kt +++ b/kotlinx-coroutines-core/concurrent/test/flow/FlowCancellationTest.kt @@ -12,7 +12,7 @@ import kotlin.test.* class FlowCancellationTest : TestBase() { @Test - fun testEmitIsCooperative() = runMtTest { + fun testEmitIsCooperative() = runTest { val latch = Channel<Unit>(1) val job = flow { expect(1) @@ -29,7 +29,7 @@ class FlowCancellationTest : TestBase() { } @Test - fun testIsActiveOnCurrentContext() = runMtTest { + fun testIsActiveOnCurrentContext() = runTest { val latch = Channel<Unit>(1) val job = flow<Unit> { expect(1) @@ -46,7 +46,7 @@ class FlowCancellationTest : TestBase() { } @Test - fun testFlowWithEmptyContext() = runMtTest { + fun testFlowWithEmptyContext() = runTest { expect(1) withEmptyContext { val flow = flow { diff --git a/kotlinx-coroutines-core/concurrent/test/flow/StateFlowCommonStressTest.kt b/kotlinx-coroutines-core/concurrent/test/flow/StateFlowCommonStressTest.kt index f2fb41a5..abd191e5 100644 --- a/kotlinx-coroutines-core/concurrent/test/flow/StateFlowCommonStressTest.kt +++ b/kotlinx-coroutines-core/concurrent/test/flow/StateFlowCommonStressTest.kt @@ -13,7 +13,7 @@ class StateFlowCommonStressTest : TestBase() { private val state = MutableStateFlow<Long>(0) @Test - fun testSingleEmitterAndCollector() = runMtTest { + fun testSingleEmitterAndCollector() = runTest { var collected = 0L val collector = launch(Dispatchers.Default) { // collect, but abort and collect again after every 1000 values to stress allocation/deallocation diff --git a/kotlinx-coroutines-core/concurrent/test/flow/StateFlowUpdateCommonTest.kt b/kotlinx-coroutines-core/concurrent/test/flow/StateFlowUpdateCommonTest.kt index 1e797094..8c75b461 100644 --- a/kotlinx-coroutines-core/concurrent/test/flow/StateFlowUpdateCommonTest.kt +++ b/kotlinx-coroutines-core/concurrent/test/flow/StateFlowUpdateCommonTest.kt @@ -21,7 +21,7 @@ class StateFlowUpdateCommonTest : TestBase() { @Test fun testGetAndUpdate() = doTest { getAndUpdate { it + 1 } } - private fun doTest(increment: MutableStateFlow<Int>.() -> Unit) = runMtTest { + private fun doTest(increment: MutableStateFlow<Int>.() -> Unit) = runTest { val flow = MutableStateFlow(0) val j1 = launch(Dispatchers.Default) { repeat(iterations / 2) { diff --git a/kotlinx-coroutines-core/concurrent/test/internal/LockFreeLinkedListTest.kt b/kotlinx-coroutines-core/concurrent/test/internal/LockFreeLinkedListTest.kt deleted file mode 100644 index 7e85d495..00000000 --- a/kotlinx-coroutines-core/concurrent/test/internal/LockFreeLinkedListTest.kt +++ /dev/null @@ -1,84 +0,0 @@ -/* - * Copyright 2016-2021 JetBrains s.r.o. Use of this source code is governed by the Apache 2.0 license. - */ - -package kotlinx.coroutines.internal - -import kotlin.test.* - -class LockFreeLinkedListTest { - data class IntNode(val i: Int) : LockFreeLinkedListNode() - - @Test - fun testSimpleAddLast() { - val list = LockFreeLinkedListHead() - assertContents(list) - val n1 = IntNode(1).apply { list.addLast(this) } - assertContents(list, 1) - val n2 = IntNode(2).apply { list.addLast(this) } - assertContents(list, 1, 2) - val n3 = IntNode(3).apply { list.addLast(this) } - assertContents(list, 1, 2, 3) - val n4 = IntNode(4).apply { list.addLast(this) } - assertContents(list, 1, 2, 3, 4) - assertTrue(n1.remove()) - assertContents(list, 2, 3, 4) - assertTrue(n3.remove()) - assertContents(list, 2, 4) - assertTrue(n4.remove()) - assertContents(list, 2) - assertTrue(n2.remove()) - assertFalse(n2.remove()) - assertContents(list) - } - - @Test - fun testCondOps() { - val list = LockFreeLinkedListHead() - assertContents(list) - assertTrue(list.addLastIf(IntNode(1)) { true }) - assertContents(list, 1) - assertFalse(list.addLastIf(IntNode(2)) { false }) - assertContents(list, 1) - assertTrue(list.addLastIf(IntNode(3)) { true }) - assertContents(list, 1, 3) - assertFalse(list.addLastIf(IntNode(4)) { false }) - assertContents(list, 1, 3) - } - - @Test - fun testAtomicOpsSingle() { - val list = LockFreeLinkedListHead() - assertContents(list) - val n1 = IntNode(1).also { single(list.describeAddLast(it)) } - assertContents(list, 1) - val n2 = IntNode(2).also { single(list.describeAddLast(it)) } - assertContents(list, 1, 2) - val n3 = IntNode(3).also { single(list.describeAddLast(it)) } - assertContents(list, 1, 2, 3) - val n4 = IntNode(4).also { single(list.describeAddLast(it)) } - assertContents(list, 1, 2, 3, 4) - } - - private fun single(part: AtomicDesc) { - val operation = object : AtomicOp<Any?>() { - init { - part.atomicOp = this - } - override fun prepare(affected: Any?): Any? = part.prepare(this) - override fun complete(affected: Any?, failure: Any?) = part.complete(this, failure) - } - assertTrue(operation.perform(null) == null) - } - - private fun assertContents(list: LockFreeLinkedListHead, vararg expected: Int) { - list.validate() - val n = expected.size - val actual = IntArray(n) - var index = 0 - list.forEach<IntNode> { actual[index++] = it.i } - assertEquals(n, index) - for (i in 0 until n) assertEquals(expected[i], actual[i], "item $i") - assertEquals(expected.isEmpty(), list.isEmpty) - } -} diff --git a/kotlinx-coroutines-core/concurrent/test/selects/SelectChannelStressTest.kt b/kotlinx-coroutines-core/concurrent/test/selects/SelectChannelStressTest.kt index 29c6c348..a5bf90ac 100644 --- a/kotlinx-coroutines-core/concurrent/test/selects/SelectChannelStressTest.kt +++ b/kotlinx-coroutines-core/concurrent/test/selects/SelectChannelStressTest.kt @@ -6,7 +6,6 @@ package kotlinx.coroutines.selects import kotlinx.coroutines.* import kotlinx.coroutines.channels.* -import kotlinx.coroutines.intrinsics.* import kotlin.test.* class SelectChannelStressTest: TestBase() { @@ -15,7 +14,7 @@ class SelectChannelStressTest: TestBase() { private val iterations = (if (isNative) 1_000 else 1_000_000) * stressTestMultiplier @Test - fun testSelectSendResourceCleanupArrayChannel() = runMtTest { + fun testSelectSendResourceCleanupBufferedChannel() = runTest { val channel = Channel<Int>(1) expect(1) channel.send(-1) // fill the buffer, so all subsequent sends cannot proceed @@ -29,7 +28,7 @@ class SelectChannelStressTest: TestBase() { } @Test - fun testSelectReceiveResourceCleanupArrayChannel() = runMtTest { + fun testSelectReceiveResourceCleanupBufferedChannel() = runTest { val channel = Channel<Int>(1) expect(1) repeat(iterations) { i -> @@ -42,7 +41,7 @@ class SelectChannelStressTest: TestBase() { } @Test - fun testSelectSendResourceCleanupRendezvousChannel() = runMtTest { + fun testSelectSendResourceCleanupRendezvousChannel() = runTest { val channel = Channel<Int>(Channel.RENDEZVOUS) expect(1) repeat(iterations) { i -> @@ -55,7 +54,7 @@ class SelectChannelStressTest: TestBase() { } @Test - fun testSelectReceiveResourceRendezvousChannel() = runMtTest { + fun testSelectReceiveResourceRendezvousChannel() = runTest { val channel = Channel<Int>(Channel.RENDEZVOUS) expect(1) repeat(iterations) { i -> @@ -67,9 +66,5 @@ class SelectChannelStressTest: TestBase() { finish(iterations + 2) } - internal fun <R> SelectBuilder<R>.default(block: suspend () -> R) { - this as SelectBuilderImpl // type assertion - if (!trySelect()) return - block.startCoroutineUnintercepted(this) - } + internal fun <R> SelectBuilder<R>.default(block: suspend () -> R) = onTimeout(0, block) } diff --git a/kotlinx-coroutines-core/concurrent/test/selects/SelectMutexStressTest.kt b/kotlinx-coroutines-core/concurrent/test/selects/SelectMutexStressTest.kt index 8f649c2f..9395a98b 100644 --- a/kotlinx-coroutines-core/concurrent/test/selects/SelectMutexStressTest.kt +++ b/kotlinx-coroutines-core/concurrent/test/selects/SelectMutexStressTest.kt @@ -28,7 +28,6 @@ class SelectMutexStressTest : TestBase() { yield() // so it can cleanup after itself } assertTrue(mutex.isLocked) - assertTrue(mutex.isLockedEmptyQueueState) finish(n + 2) } -}
\ No newline at end of file +} diff --git a/kotlinx-coroutines-core/concurrent/test/sync/MutexStressTest.kt b/kotlinx-coroutines-core/concurrent/test/sync/MutexStressTest.kt index 73b62aee..77fa7bf1 100644 --- a/kotlinx-coroutines-core/concurrent/test/sync/MutexStressTest.kt +++ b/kotlinx-coroutines-core/concurrent/test/sync/MutexStressTest.kt @@ -14,24 +14,24 @@ class MutexStressTest : TestBase() { private val n = (if (isNative) 1_000 else 10_000) * stressTestMultiplier @Test - fun testDefaultDispatcher() = runMtTest { testBody(Dispatchers.Default) } + fun testDefaultDispatcher() = runTest { testBody(Dispatchers.Default) } @Test - fun testSingleThreadContext() = runMtTest { + fun testSingleThreadContext() = runTest { newSingleThreadContext("testSingleThreadContext").use { testBody(it) } } @Test - fun testMultiThreadedContextWithSingleWorker() = runMtTest { + fun testMultiThreadedContextWithSingleWorker() = runTest { newFixedThreadPoolContext(1, "testMultiThreadedContextWithSingleWorker").use { testBody(it) } } @Test - fun testMultiThreadedContext() = runMtTest { + fun testMultiThreadedContext() = runTest { newFixedThreadPoolContext(8, "testMultiThreadedContext").use { testBody(it) } @@ -56,7 +56,7 @@ class MutexStressTest : TestBase() { } @Test - fun stressUnlockCancelRace() = runMtTest { + fun stressUnlockCancelRace() = runTest { val n = 10_000 * stressTestMultiplier val mutex = Mutex(true) // create a locked mutex newSingleThreadContext("SemaphoreStressTest").use { pool -> @@ -86,7 +86,7 @@ class MutexStressTest : TestBase() { } @Test - fun stressUnlockCancelRaceWithSelect() = runMtTest { + fun stressUnlockCancelRaceWithSelect() = runTest { val n = 10_000 * stressTestMultiplier val mutex = Mutex(true) // create a locked mutex newSingleThreadContext("SemaphoreStressTest").use { pool -> @@ -119,7 +119,7 @@ class MutexStressTest : TestBase() { } @Test - fun testShouldBeUnlockedOnCancellation() = runMtTest { + fun testShouldBeUnlockedOnCancellation() = runTest { val mutex = Mutex() val n = 1000 * stressTestMultiplier repeat(n) { diff --git a/kotlinx-coroutines-core/concurrent/test/sync/SemaphoreStressTest.kt b/kotlinx-coroutines-core/concurrent/test/sync/SemaphoreStressTest.kt index c5f20389..9a3d25bc 100644 --- a/kotlinx-coroutines-core/concurrent/test/sync/SemaphoreStressTest.kt +++ b/kotlinx-coroutines-core/concurrent/test/sync/SemaphoreStressTest.kt @@ -13,7 +13,7 @@ class SemaphoreStressTest : TestBase() { private val iterations = (if (isNative) 1_000 else 10_000) * stressTestMultiplier @Test - fun testStressTestAsMutex() = runMtTest { + fun testStressTestAsMutex() = runTest { val n = iterations val k = 100 var shared = 0 @@ -32,7 +32,7 @@ class SemaphoreStressTest : TestBase() { } @Test - fun testStress() = runMtTest { + fun testStress() = runTest { val n = iterations val k = 100 val semaphore = Semaphore(10) @@ -48,7 +48,7 @@ class SemaphoreStressTest : TestBase() { } @Test - fun testStressAsMutex() = runMtTest { + fun testStressAsMutex() = runTest { runBlocking(Dispatchers.Default) { val n = iterations val k = 100 @@ -69,7 +69,7 @@ class SemaphoreStressTest : TestBase() { } @Test - fun testStressCancellation() = runMtTest { + fun testStressCancellation() = runTest { val n = iterations val semaphore = Semaphore(1) semaphore.acquire() @@ -90,7 +90,7 @@ class SemaphoreStressTest : TestBase() { * the semaphore into an incorrect state where permits are leaked. */ @Test - fun testStressReleaseCancelRace() = runMtTest { + fun testStressReleaseCancelRace() = runTest { val n = iterations val semaphore = Semaphore(1, 1) newSingleThreadContext("SemaphoreStressTest").use { pool -> @@ -120,7 +120,7 @@ class SemaphoreStressTest : TestBase() { } @Test - fun testShouldBeUnlockedOnCancellation() = runMtTest { + fun testShouldBeUnlockedOnCancellation() = runTest { val semaphore = Semaphore(1) val n = 1000 * stressTestMultiplier repeat(n) { diff --git a/integration/kotlinx-coroutines-jdk8/src/future/Future.kt b/kotlinx-coroutines-core/jdk8/src/future/Future.kt index caf2a3c3..f7b4fdca 100644 --- a/integration/kotlinx-coroutines-jdk8/src/future/Future.kt +++ b/kotlinx-coroutines-core/jdk8/src/future/Future.kt @@ -40,7 +40,7 @@ public fun <T> CoroutineScope.future( val newContext = this.newCoroutineContext(context) val future = CompletableFuture<T>() val coroutine = CompletableFutureCoroutine(newContext, future) - future.whenComplete(coroutine) // Cancel coroutine if future was completed externally + future.handle(coroutine) // Cancel coroutine if future was completed externally coroutine.start(start, coroutine, block) return future } @@ -48,8 +48,8 @@ public fun <T> CoroutineScope.future( private class CompletableFutureCoroutine<T>( context: CoroutineContext, private val future: CompletableFuture<T> -) : AbstractCoroutine<T>(context, initParentJob = true, active = true), BiConsumer<T?, Throwable?> { - override fun accept(value: T?, exception: Throwable?) { +) : AbstractCoroutine<T>(context, initParentJob = true, active = true), BiFunction<T?, Throwable?, Unit> { + override fun apply(value: T?, exception: Throwable?) { cancel() } @@ -58,10 +58,12 @@ private class CompletableFutureCoroutine<T>( } override fun onCancelled(cause: Throwable, handled: Boolean) { - if (!future.completeExceptionally(cause) && !handled) { - // prevents loss of exception that was not handled by parent & could not be set to CompletableFuture - handleCoroutineException(context, cause) - } + /* + * Here we can potentially lose the cause if the failure is racing with future's + * external cancellation. We are consistent with other future implementations + * (LF, FT, CF) and give up on such exception. + */ + future.completeExceptionally(cause) } } @@ -97,7 +99,7 @@ public fun Job.asCompletableFuture(): CompletableFuture<Unit> { } private fun Job.setupCancellation(future: CompletableFuture<*>) { - future.whenComplete { _, exception -> + future.handle { _, exception -> cancel(exception?.let { it as? CancellationException ?: CancellationException("CompletableFuture was completed exceptionally", it) }) @@ -125,7 +127,7 @@ public fun <T> CompletionStage<T>.asDeferred(): Deferred<T> { } } val result = CompletableDeferred<T>() - whenComplete { value, exception -> + handle { value, exception -> try { if (exception == null) { // the future has completed normally @@ -168,8 +170,8 @@ public suspend fun <T> CompletionStage<T>.await(): T { } // slow path -- suspend return suspendCancellableCoroutine { cont: CancellableContinuation<T> -> - val consumer = ContinuationConsumer(cont) - whenComplete(consumer) + val consumer = ContinuationHandler(cont) + handle(consumer) cont.invokeOnCancellation { future.cancel(false) consumer.cont = null // shall clear reference to continuation to aid GC @@ -177,11 +179,11 @@ public suspend fun <T> CompletionStage<T>.await(): T { } } -private class ContinuationConsumer<T>( +private class ContinuationHandler<T>( @Volatile @JvmField var cont: Continuation<T>? -) : BiConsumer<T?, Throwable?> { +) : BiFunction<T?, Throwable?, Unit> { @Suppress("UNCHECKED_CAST") - override fun accept(result: T?, exception: Throwable?) { + override fun apply(result: T?, exception: Throwable?) { val cont = this.cont ?: return // atomically read current value unless null if (exception == null) { // the future has completed normally diff --git a/integration/kotlinx-coroutines-jdk8/src/stream/Stream.kt b/kotlinx-coroutines-core/jdk8/src/stream/Stream.kt index b0d72de8..b0d72de8 100644 --- a/integration/kotlinx-coroutines-jdk8/src/stream/Stream.kt +++ b/kotlinx-coroutines-core/jdk8/src/stream/Stream.kt diff --git a/integration/kotlinx-coroutines-jdk8/src/time/Time.kt b/kotlinx-coroutines-core/jdk8/src/time/Time.kt index 78cf6e53..78cf6e53 100644 --- a/integration/kotlinx-coroutines-jdk8/src/time/Time.kt +++ b/kotlinx-coroutines-core/jdk8/src/time/Time.kt diff --git a/kotlinx-coroutines-core/js/src/CoroutineExceptionHandlerImpl.kt b/kotlinx-coroutines-core/js/src/CoroutineExceptionHandlerImpl.kt deleted file mode 100644 index 54a65e10..00000000 --- a/kotlinx-coroutines-core/js/src/CoroutineExceptionHandlerImpl.kt +++ /dev/null @@ -1,12 +0,0 @@ -/* - * Copyright 2016-2021 JetBrains s.r.o. Use of this source code is governed by the Apache 2.0 license. - */ - -package kotlinx.coroutines - -import kotlin.coroutines.* - -internal actual fun handleCoroutineExceptionImpl(context: CoroutineContext, exception: Throwable) { - // log exception - console.error(exception) -} diff --git a/kotlinx-coroutines-core/js/src/Dispatchers.kt b/kotlinx-coroutines-core/js/src/Dispatchers.kt index 3eec5408..1304c5a9 100644 --- a/kotlinx-coroutines-core/js/src/Dispatchers.kt +++ b/kotlinx-coroutines-core/js/src/Dispatchers.kt @@ -19,11 +19,6 @@ public actual object Dispatchers { internal fun injectMain(dispatcher: MainCoroutineDispatcher) { injectedMainDispatcher = dispatcher } - - @PublishedApi - internal fun resetInjectedMain() { - injectedMainDispatcher = null - } } private class JsMainDispatcher( diff --git a/kotlinx-coroutines-core/js/src/JSDispatcher.kt b/kotlinx-coroutines-core/js/src/JSDispatcher.kt index 603005d5..8ddb9033 100644 --- a/kotlinx-coroutines-core/js/src/JSDispatcher.kt +++ b/kotlinx-coroutines-core/js/src/JSDispatcher.kt @@ -47,7 +47,6 @@ internal sealed class SetTimeoutBasedDispatcher: CoroutineDispatcher(), Delay { override fun scheduleResumeAfterDelay(timeMillis: Long, continuation: CancellableContinuation<Unit>) { val handle = setTimeout({ with(continuation) { resumeUndispatched(Unit) } }, delayToInt(timeMillis)) - // Actually on cancellation, but clearTimeout is idempotent continuation.invokeOnCancellation(handler = ClearTimeout(handle).asHandler) } } @@ -64,7 +63,7 @@ internal object SetTimeoutDispatcher : SetTimeoutBasedDispatcher() { } } -private class ClearTimeout(private val handle: Int) : CancelHandler(), DisposableHandle { +private open class ClearTimeout(protected val handle: Int) : CancelHandler(), DisposableHandle { override fun dispose() { clearTimeout(handle) @@ -83,15 +82,18 @@ internal class WindowDispatcher(private val window: Window) : CoroutineDispatche override fun dispatch(context: CoroutineContext, block: Runnable) = queue.enqueue(block) override fun scheduleResumeAfterDelay(timeMillis: Long, continuation: CancellableContinuation<Unit>) { - window.setTimeout({ with(continuation) { resumeUndispatched(Unit) } }, delayToInt(timeMillis)) + val handle = window.setTimeout({ with(continuation) { resumeUndispatched(Unit) } }, delayToInt(timeMillis)) + continuation.invokeOnCancellation(handler = WindowClearTimeout(handle).asHandler) } override fun invokeOnTimeout(timeMillis: Long, block: Runnable, context: CoroutineContext): DisposableHandle { val handle = window.setTimeout({ block.run() }, delayToInt(timeMillis)) - return object : DisposableHandle { - override fun dispose() { - window.clearTimeout(handle) - } + return WindowClearTimeout(handle) + } + + private inner class WindowClearTimeout(handle: Int) : ClearTimeout(handle) { + override fun dispose() { + window.clearTimeout(handle) } } } @@ -129,7 +131,7 @@ private class WindowMessageQueue(private val window: Window) : MessageQueue() { * * Yet there could be a long tail of "slow" reschedules, but it should be amortized by the queue size. */ -internal abstract class MessageQueue : ArrayQueue<Runnable>() { +internal abstract class MessageQueue : MutableList<Runnable> by ArrayDeque() { val yieldEvery = 16 // yield to JS macrotask event loop after this many processed messages private var scheduled = false @@ -138,7 +140,7 @@ internal abstract class MessageQueue : ArrayQueue<Runnable>() { abstract fun reschedule() fun enqueue(element: Runnable) { - addLast(element) + add(element) if (!scheduled) { scheduled = true schedule() @@ -153,7 +155,7 @@ internal abstract class MessageQueue : ArrayQueue<Runnable>() { element.run() } } finally { - if (isEmpty) { + if (isEmpty()) { scheduled = false } else { reschedule() diff --git a/kotlinx-coroutines-core/js/src/Window.kt b/kotlinx-coroutines-core/js/src/Window.kt index dad0c04b..7e993283 100644 --- a/kotlinx-coroutines-core/js/src/Window.kt +++ b/kotlinx-coroutines-core/js/src/Window.kt @@ -4,7 +4,6 @@ package kotlinx.coroutines -import kotlinx.coroutines.internal.* import org.w3c.dom.Window /** @@ -35,8 +34,8 @@ private fun Window.asWindowAnimationQueue(): WindowAnimationQueue = private class WindowAnimationQueue(private val window: Window) { private val dispatcher = window.asCoroutineDispatcher() private var scheduled = false - private var current = ArrayQueue<CancellableContinuation<Double>>() - private var next = ArrayQueue<CancellableContinuation<Double>>() + private var current = ArrayDeque<CancellableContinuation<Double>>() + private var next = ArrayDeque<CancellableContinuation<Double>>() private var timestamp = 0.0 fun enqueue(cont: CancellableContinuation<Double>) { diff --git a/kotlinx-coroutines-core/js/src/internal/Concurrent.kt b/kotlinx-coroutines-core/js/src/internal/Concurrent.kt index 71f65227..6272679e 100644 --- a/kotlinx-coroutines-core/js/src/internal/Concurrent.kt +++ b/kotlinx-coroutines-core/js/src/internal/Concurrent.kt @@ -13,7 +13,5 @@ internal class NoOpLock { fun unlock(): Unit {} } -internal actual fun <E> subscriberList(): SubscribersList<E> = CopyOnWriteList() - internal actual fun <E> identitySet(expectedSize: Int): MutableSet<E> = HashSet(expectedSize) diff --git a/kotlinx-coroutines-core/js/src/internal/CoroutineExceptionHandlerImpl.kt b/kotlinx-coroutines-core/js/src/internal/CoroutineExceptionHandlerImpl.kt new file mode 100644 index 00000000..675cc4a6 --- /dev/null +++ b/kotlinx-coroutines-core/js/src/internal/CoroutineExceptionHandlerImpl.kt @@ -0,0 +1,26 @@ +/* + * Copyright 2016-2022 JetBrains s.r.o. Use of this source code is governed by the Apache 2.0 license. + */ + +package kotlinx.coroutines.internal + +import kotlinx.coroutines.* +import kotlin.coroutines.* + +private val platformExceptionHandlers_ = mutableSetOf<CoroutineExceptionHandler>() + +internal actual val platformExceptionHandlers: Collection<CoroutineExceptionHandler> + get() = platformExceptionHandlers_ + +internal actual fun ensurePlatformExceptionHandlerLoaded(callback: CoroutineExceptionHandler) { + platformExceptionHandlers_ += callback +} + +internal actual fun propagateExceptionFinalResort(exception: Throwable) { + // log exception + console.error(exception) +} + +internal actual class DiagnosticCoroutineContextException actual constructor(context: CoroutineContext) : + RuntimeException(context.toString()) + diff --git a/kotlinx-coroutines-core/js/src/internal/LinkedList.kt b/kotlinx-coroutines-core/js/src/internal/LinkedList.kt index d8c07f4e..de5d4911 100644 --- a/kotlinx-coroutines-core/js/src/internal/LinkedList.kt +++ b/kotlinx-coroutines-core/js/src/internal/LinkedList.kt @@ -6,6 +6,8 @@ package kotlinx.coroutines.internal +import kotlinx.coroutines.* + private typealias Node = LinkedListNode /** @suppress **This is unstable API and it is subject to change.** */ @Suppress("NO_ACTUAL_CLASS_MEMBER_FOR_EXPECTED_CLASS") // :TODO: Remove when fixed: https://youtrack.jetbrains.com/issue/KT-23703 @@ -15,7 +17,7 @@ public actual typealias LockFreeLinkedListNode = LinkedListNode public actual typealias LockFreeLinkedListHead = LinkedListHead /** @suppress **This is unstable API and it is subject to change.** */ -public open class LinkedListNode { +public open class LinkedListNode : DisposableHandle { @PublishedApi internal var _next = this @PublishedApi internal var _prev = this @PublishedApi internal var _removed: Boolean = false @@ -42,6 +44,10 @@ public open class LinkedListNode { return removeImpl() } + override fun dispose() { + remove() + } + @PublishedApi internal fun removeImpl(): Boolean { if (_removed) return false @@ -90,75 +96,6 @@ public open class LinkedListNode { check(next.removeImpl()) { "Should remove" } return next } - - public inline fun <reified T> removeFirstIfIsInstanceOfOrPeekIf(predicate: (T) -> Boolean): T? { - val next = _next - if (next === this) return null - if (next !is T) return null - if (predicate(next)) return next - check(next.removeImpl()) { "Should remove" } - return next - } -} - -/** @suppress **This is unstable API and it is subject to change.** */ -public actual open class AddLastDesc<T : Node> actual constructor( - actual val queue: Node, - actual val node: T -) : AbstractAtomicDesc() { - override val affectedNode: Node get() = queue._prev - actual override fun finishPrepare(prepareOp: PrepareOp) {} - override fun onComplete() = queue.addLast(node) - actual override fun finishOnSuccess(affected: LockFreeLinkedListNode, next: LockFreeLinkedListNode) = Unit -} - -/** @suppress **This is unstable API and it is subject to change.** */ -public actual open class RemoveFirstDesc<T> actual constructor( - actual val queue: LockFreeLinkedListNode -) : AbstractAtomicDesc() { - @Suppress("UNCHECKED_CAST") - actual val result: T get() = affectedNode as T - override val affectedNode: Node = queue.nextNode - actual override fun finishPrepare(prepareOp: PrepareOp) {} - override fun onComplete() { queue.removeFirstOrNull() } - actual override fun finishOnSuccess(affected: LockFreeLinkedListNode, next: LockFreeLinkedListNode) = Unit -} - -/** @suppress **This is unstable API and it is subject to change.** */ -public actual abstract class AbstractAtomicDesc : AtomicDesc() { - protected abstract val affectedNode: Node - actual abstract fun finishPrepare(prepareOp: PrepareOp) - protected abstract fun onComplete() - - actual open fun onPrepare(prepareOp: PrepareOp): Any? { - finishPrepare(prepareOp) - return null - } - - actual open fun onRemoved(affected: Node) {} - - actual final override fun prepare(op: AtomicOp<*>): Any? { - val affected = affectedNode - val failure = failure(affected) - if (failure != null) return failure - @Suppress("UNCHECKED_CAST") - return onPrepare(PrepareOp(affected, this, op)) - } - - actual final override fun complete(op: AtomicOp<*>, failure: Any?) = onComplete() - protected actual open fun failure(affected: LockFreeLinkedListNode): Any? = null // Never fails by default - protected actual open fun retry(affected: LockFreeLinkedListNode, next: Any): Boolean = false // Always succeeds - protected actual abstract fun finishOnSuccess(affected: LockFreeLinkedListNode, next: LockFreeLinkedListNode) -} - -/** @suppress **This is unstable API and it is subject to change.** */ -public actual class PrepareOp( - actual val affected: LockFreeLinkedListNode, - actual val desc: AbstractAtomicDesc, - actual override val atomicOp: AtomicOp<*> -): OpDescriptor() { - override fun perform(affected: Any?): Any? = null - actual fun finishPrepare() {} } /** @suppress **This is unstable API and it is subject to change.** */ diff --git a/kotlinx-coroutines-core/js/src/internal/Synchronized.kt b/kotlinx-coroutines-core/js/src/internal/Synchronized.kt index dcbb2021..05db5285 100644 --- a/kotlinx-coroutines-core/js/src/internal/Synchronized.kt +++ b/kotlinx-coroutines-core/js/src/internal/Synchronized.kt @@ -16,5 +16,4 @@ public actual typealias SynchronizedObject = Any * @suppress **This an internal API and should not be used from general code.** */ @InternalCoroutinesApi -public actual inline fun <T> synchronized(lock: SynchronizedObject, block: () -> T): T = - block() +public actual inline fun <T> synchronizedImpl(lock: SynchronizedObject, block: () -> T): T = block() diff --git a/kotlinx-coroutines-core/js/src/internal/ThreadLocal.kt b/kotlinx-coroutines-core/js/src/internal/ThreadLocal.kt index e1825d67..c8dd0968 100644 --- a/kotlinx-coroutines-core/js/src/internal/ThreadLocal.kt +++ b/kotlinx-coroutines-core/js/src/internal/ThreadLocal.kt @@ -4,9 +4,11 @@ package kotlinx.coroutines.internal -internal actual class CommonThreadLocal<T> actual constructor() { +internal actual class CommonThreadLocal<T> { private var value: T? = null @Suppress("UNCHECKED_CAST") actual fun get(): T = value as T actual fun set(value: T) { this.value = value } } + +internal actual fun<T> commonThreadLocal(name: Symbol): CommonThreadLocal<T> = CommonThreadLocal() diff --git a/kotlinx-coroutines-core/js/test/MessageQueueTest.kt b/kotlinx-coroutines-core/js/test/MessageQueueTest.kt index de514c76..7ce73804 100644 --- a/kotlinx-coroutines-core/js/test/MessageQueueTest.kt +++ b/kotlinx-coroutines-core/js/test/MessageQueueTest.kt @@ -36,41 +36,41 @@ class MessageQueueTest { @Test fun testBasic() { - assertTrue(queue.isEmpty) + assertTrue(queue.isEmpty()) queue.enqueue(Box(1)) - assertFalse(queue.isEmpty) + assertFalse(queue.isEmpty()) assertTrue(scheduled) queue.enqueue(Box(2)) - assertFalse(queue.isEmpty) + assertFalse(queue.isEmpty()) scheduled = false queue.process() assertEquals(listOf(1, 2), processed) assertFalse(scheduled) - assertTrue(queue.isEmpty) + assertTrue(queue.isEmpty()) } @Test fun testRescheduleFromProcess() { - assertTrue(queue.isEmpty) + assertTrue(queue.isEmpty()) queue.enqueue(ReBox(1)) - assertFalse(queue.isEmpty) + assertFalse(queue.isEmpty()) assertTrue(scheduled) queue.enqueue(ReBox(2)) - assertFalse(queue.isEmpty) + assertFalse(queue.isEmpty()) scheduled = false queue.process() assertEquals(listOf(1, 2, 11, 12), processed) assertFalse(scheduled) - assertTrue(queue.isEmpty) + assertTrue(queue.isEmpty()) } @Test fun testResizeAndWrap() { repeat(10) { phase -> val n = 10 * (phase + 1) - assertTrue(queue.isEmpty) + assertTrue(queue.isEmpty()) repeat(n) { queue.enqueue(Box(it)) - assertFalse(queue.isEmpty) + assertFalse(queue.isEmpty()) assertTrue(scheduled) } var countYields = 0 @@ -84,4 +84,4 @@ class MessageQueueTest { processed.clear() } } -}
\ No newline at end of file +} diff --git a/kotlinx-coroutines-core/js/test/TestBase.kt b/kotlinx-coroutines-core/js/test/TestBase.kt index c930c200..f0e3a2dc 100644 --- a/kotlinx-coroutines-core/js/test/TestBase.kt +++ b/kotlinx-coroutines-core/js/test/TestBase.kt @@ -138,3 +138,5 @@ public actual open class TestBase actual constructor() { private fun <T> Promise<T>.finally(block: () -> Unit): Promise<T> = then(onFulfilled = { value -> block(); value }, onRejected = { ex -> block(); throw ex }) + +public actual val isJavaAndWindows: Boolean get() = false diff --git a/kotlinx-coroutines-core/jvm/resources/DebugProbesKt.bin b/kotlinx-coroutines-core/jvm/resources/DebugProbesKt.bin Binary files differindex a6ea0a40..9d171f3a 100644 --- a/kotlinx-coroutines-core/jvm/resources/DebugProbesKt.bin +++ b/kotlinx-coroutines-core/jvm/resources/DebugProbesKt.bin diff --git a/kotlinx-coroutines-core/jvm/src/CoroutineContext.kt b/kotlinx-coroutines-core/jvm/src/CoroutineContext.kt index 7209bee8..b736125d 100644 --- a/kotlinx-coroutines-core/jvm/src/CoroutineContext.kt +++ b/kotlinx-coroutines-core/jvm/src/CoroutineContext.kt @@ -167,9 +167,11 @@ internal actual class UndispatchedCoroutine<in T>actual constructor ( uCont: Continuation<T> ) : ScopeCoroutine<T>(if (context[UndispatchedMarker] == null) context + UndispatchedMarker else context, uCont) { - /* - * The state is thread-local because this coroutine can be used concurrently. - * Scenario of usage (withContinuationContext): + /** + * The state of [ThreadContextElement]s associated with the current undispatched coroutine. + * It is stored in a thread local because this coroutine can be used concurrently in suspend-resume race scenario. + * See the followin, boiled down example with inlined `withContinuationContext` body: + * ``` * val state = saveThreadContext(ctx) * try { * invokeSmthWithThisCoroutineAsCompletion() // Completion implies that 'afterResume' will be called @@ -178,8 +180,40 @@ internal actual class UndispatchedCoroutine<in T>actual constructor ( * thisCoroutine().clearThreadContext() // Concurrently the "smth" could've been already resumed on a different thread * // and it also calls saveThreadContext and clearThreadContext * } + * ``` + * + * Usage note: + * + * This part of the code is performance-sensitive. + * It is a well-established pattern to wrap various activities into system-specific undispatched + * `withContext` for the sake of logging, MDC, tracing etc., meaning that there exists thousands of + * undispatched coroutines. + * Each access to Java's [ThreadLocal] leaves a footprint in the corresponding Thread's `ThreadLocalMap` + * that is cleared automatically as soon as the associated thread-local (-> UndispatchedCoroutine) is garbage collected. + * When such coroutines are promoted to old generation, `ThreadLocalMap`s become bloated and an arbitrary accesses to thread locals + * start to consume significant amount of CPU because these maps are open-addressed and cleaned up incrementally on each access. + * (You can read more about this effect as "GC nepotism"). + * + * To avoid that, we attempt to narrow down the lifetime of this thread local as much as possible: + * * It's never accessed when we are sure there are no thread context elements + * * It's cleaned up via [ThreadLocal.remove] as soon as the coroutine is suspended or finished. + */ + private val threadStateToRecover = ThreadLocal<Pair<CoroutineContext, Any?>>() + + /* + * Indicates that a coroutine has at least one thread context element associated with it + * and that 'threadStateToRecover' is going to be set in case of dispatchhing in order to preserve them. + * Better than nullable thread-local for easier debugging. + * + * It is used as a performance optimization to avoid 'threadStateToRecover' initialization + * (note: tl.get() initializes thread local), + * and is prone to false-positives as it is never reset: otherwise + * it may lead to logical data races between suspensions point where + * coroutine is yet being suspended in one thread while already being resumed + * in another. */ - private var threadStateToRecover = ThreadLocal<Pair<CoroutineContext, Any?>>() + @Volatile + private var threadLocalIsSet = false init { /* @@ -213,19 +247,22 @@ internal actual class UndispatchedCoroutine<in T>actual constructor ( } fun saveThreadContext(context: CoroutineContext, oldValue: Any?) { + threadLocalIsSet = true // Specify that thread-local is touched at all threadStateToRecover.set(context to oldValue) } fun clearThreadContext(): Boolean { - if (threadStateToRecover.get() == null) return false - threadStateToRecover.set(null) - return true + return !(threadLocalIsSet && threadStateToRecover.get() == null).also { + threadStateToRecover.remove() + } } override fun afterResume(state: Any?) { - threadStateToRecover.get()?.let { (ctx, value) -> - restoreThreadContext(ctx, value) - threadStateToRecover.set(null) + if (threadLocalIsSet) { + threadStateToRecover.get()?.let { (ctx, value) -> + restoreThreadContext(ctx, value) + } + threadStateToRecover.remove() } // resume undispatched -- update context but stay on the same dispatcher val result = recoverResult(state, uCont) @@ -245,9 +282,12 @@ internal actual val CoroutineContext.coroutineName: String? get() { private const val DEBUG_THREAD_NAME_SEPARATOR = " @" @IgnoreJreRequirement // desugared hashcode implementation +@PublishedApi internal data class CoroutineId( + // Used by the IDEA debugger via reflection and must be kept binary-compatible, see KTIJ-24102 val id: Long ) : ThreadContextElement<String>, AbstractCoroutineContextElement(CoroutineId) { + // Used by the IDEA debugger via reflection and must be kept binary-compatible, see KTIJ-24102 companion object Key : CoroutineContext.Key<CoroutineId> override fun toString(): String = "CoroutineId($id)" diff --git a/kotlinx-coroutines-core/jvm/src/CoroutineExceptionHandlerImpl.kt b/kotlinx-coroutines-core/jvm/src/CoroutineExceptionHandlerImpl.kt deleted file mode 100644 index 4259092e..00000000 --- a/kotlinx-coroutines-core/jvm/src/CoroutineExceptionHandlerImpl.kt +++ /dev/null @@ -1,62 +0,0 @@ -/* - * Copyright 2016-2021 JetBrains s.r.o. Use of this source code is governed by the Apache 2.0 license. - */ - -package kotlinx.coroutines - -import java.util.* -import kotlin.coroutines.* - -/** - * A list of globally installed [CoroutineExceptionHandler] instances. - * - * Note that Android may have dummy [Thread.contextClassLoader] which is used by one-argument [ServiceLoader.load] function, - * see (https://stackoverflow.com/questions/13407006/android-class-loader-may-fail-for-processes-that-host-multiple-applications). - * So here we explicitly use two-argument `load` with a class-loader of [CoroutineExceptionHandler] class. - * - * We are explicitly using the `ServiceLoader.load(MyClass::class.java, MyClass::class.java.classLoader).iterator()` - * form of the ServiceLoader call to enable R8 optimization when compiled on Android. - */ -private val handlers: List<CoroutineExceptionHandler> = ServiceLoader.load( - CoroutineExceptionHandler::class.java, - CoroutineExceptionHandler::class.java.classLoader -).iterator().asSequence().toList() - -/** - * Private exception without stacktrace that is added to suppressed exceptions of the original exception - * when it is reported to the last-ditch current thread 'uncaughtExceptionHandler'. - * - * The purpose of this exception is to add an otherwise inaccessible diagnostic information and to - * be able to poke the failing coroutine context in the debugger. - */ -private class DiagnosticCoroutineContextException(private val context: CoroutineContext) : RuntimeException() { - override fun getLocalizedMessage(): String { - return context.toString() - } - - override fun fillInStackTrace(): Throwable { - // Prevent Android <= 6.0 bug, #1866 - stackTrace = emptyArray() - return this - } -} - -internal actual fun handleCoroutineExceptionImpl(context: CoroutineContext, exception: Throwable) { - // use additional extension handlers - for (handler in handlers) { - try { - handler.handleException(context, exception) - } catch (t: Throwable) { - // Use thread's handler if custom handler failed to handle exception - val currentThread = Thread.currentThread() - currentThread.uncaughtExceptionHandler.uncaughtException(currentThread, handlerException(exception, t)) - } - } - - // use thread's handler - val currentThread = Thread.currentThread() - // addSuppressed is never user-defined and cannot normally throw with the only exception being OOM - // we do ignore that just in case to definitely deliver the exception - runCatching { exception.addSuppressed(DiagnosticCoroutineContextException(context)) } - currentThread.uncaughtExceptionHandler.uncaughtException(currentThread, exception) -} diff --git a/kotlinx-coroutines-core/jvm/src/Debug.kt b/kotlinx-coroutines-core/jvm/src/Debug.kt index 911914a0..4a821a14 100644 --- a/kotlinx-coroutines-core/jvm/src/Debug.kt +++ b/kotlinx-coroutines-core/jvm/src/Debug.kt @@ -59,7 +59,7 @@ public const val DEBUG_PROPERTY_VALUE_AUTO: String = "auto" public const val DEBUG_PROPERTY_VALUE_ON: String = "on" /** - * Debug turned on value for [DEBUG_PROPERTY_NAME]. + * Debug turned off value for [DEBUG_PROPERTY_NAME]. */ public const val DEBUG_PROPERTY_VALUE_OFF: String = "off" diff --git a/kotlinx-coroutines-core/jvm/src/Dispatchers.kt b/kotlinx-coroutines-core/jvm/src/Dispatchers.kt index 251a567c..2222f0da 100644 --- a/kotlinx-coroutines-core/jvm/src/Dispatchers.kt +++ b/kotlinx-coroutines-core/jvm/src/Dispatchers.kt @@ -1,9 +1,7 @@ /* - * Copyright 2016-2021 JetBrains s.r.o. Use of this source code is governed by the Apache 2.0 license. + * Copyright 2016-2023 JetBrains s.r.o. Use of this source code is governed by the Apache 2.0 license. */ -@file:Suppress("unused") - package kotlinx.coroutines import kotlinx.coroutines.internal.* @@ -19,76 +17,12 @@ public const val IO_PARALLELISM_PROPERTY_NAME: String = "kotlinx.coroutines.io.p * Groups various implementations of [CoroutineDispatcher]. */ public actual object Dispatchers { - /** - * The default [CoroutineDispatcher] that is used by all standard builders like - * [launch][CoroutineScope.launch], [async][CoroutineScope.async], etc. - * if no dispatcher nor any other [ContinuationInterceptor] is specified in their context. - * - * It is backed by a shared pool of threads on JVM. By default, the maximal level of parallelism used - * by this dispatcher is equal to the number of CPU cores, but is at least two. - * Level of parallelism X guarantees that no more than X tasks can be executed in this dispatcher in parallel. - */ @JvmStatic public actual val Default: CoroutineDispatcher = DefaultScheduler - /** - * A coroutine dispatcher that is confined to the Main thread operating with UI objects. - * This dispatcher can be used either directly or via [MainScope] factory. - * Usually such dispatcher is single-threaded. - * - * Access to this property may throw [IllegalStateException] if no main thread dispatchers are present in the classpath. - * - * Depending on platform and classpath it can be mapped to different dispatchers: - * - On JS and Native it is equivalent of [Default] dispatcher. - * - On JVM it is either Android main thread dispatcher, JavaFx or Swing EDT dispatcher. It is chosen by - * [`ServiceLoader`](https://docs.oracle.com/javase/8/docs/api/java/util/ServiceLoader.html). - * - * In order to work with `Main` dispatcher, the following artifacts should be added to project runtime dependencies: - * - `kotlinx-coroutines-android` for Android Main thread dispatcher - * - `kotlinx-coroutines-javafx` for JavaFx Application thread dispatcher - * - `kotlinx-coroutines-swing` for Swing EDT dispatcher - * - * In order to set a custom `Main` dispatcher for testing purposes, add the `kotlinx-coroutines-test` artifact to - * project test dependencies. - * - * Implementation note: [MainCoroutineDispatcher.immediate] is not supported on Native and JS platforms. - */ @JvmStatic public actual val Main: MainCoroutineDispatcher get() = MainDispatcherLoader.dispatcher - /** - * A coroutine dispatcher that is not confined to any specific thread. - * It executes initial continuation of the coroutine in the current call-frame - * and lets the coroutine resume in whatever thread that is used by the corresponding suspending function, without - * mandating any specific threading policy. Nested coroutines launched in this dispatcher form an event-loop to avoid - * stack overflows. - * - * ### Event loop - * Event loop semantics is a purely internal concept and have no guarantees on the order of execution - * except that all queued coroutines will be executed on the current thread in the lexical scope of the outermost - * unconfined coroutine. - * - * For example, the following code: - * ``` - * withContext(Dispatchers.Unconfined) { - * println(1) - * withContext(Dispatchers.Unconfined) { // Nested unconfined - * println(2) - * } - * println(3) - * } - * println("Done") - * ``` - * Can print both "1 2 3" and "1 3 2", this is an implementation detail that can be changed. - * But it is guaranteed that "Done" will be printed only when both `withContext` are completed. - * - * - * Note that if you need your coroutine to be confined to a particular thread or a thread-pool after resumption, - * but still want to execute it in the current call-frame until its first suspension, then you can use - * an optional [CoroutineStart] parameter in coroutine builders like - * [launch][CoroutineScope.launch] and [async][CoroutineScope.async] setting it to - * the value of [CoroutineStart.UNDISPATCHED]. - */ @JvmStatic public actual val Unconfined: CoroutineDispatcher = kotlinx.coroutines.Unconfined @@ -161,3 +95,13 @@ public actual object Dispatchers { DefaultScheduler.shutdown() } } + +/** + * `actual` counterpart of the corresponding `expect` declaration. + * Should never be used directly from JVM sources, all accesses + * to `Dispatchers.IO` should be resolved to the corresponding member of [Dispatchers] object. + * @suppress + */ +@Suppress("EXTENSION_SHADOWED_BY_MEMBER") +@Deprecated(message = "Should not be used directly", level = DeprecationLevel.HIDDEN) +public actual val Dispatchers.IO: CoroutineDispatcher get() = Dispatchers.IO diff --git a/kotlinx-coroutines-core/jvm/src/EventLoop.kt b/kotlinx-coroutines-core/jvm/src/EventLoop.kt index 1ee651aa..7d1078cf 100644 --- a/kotlinx-coroutines-core/jvm/src/EventLoop.kt +++ b/kotlinx-coroutines-core/jvm/src/EventLoop.kt @@ -4,6 +4,10 @@ package kotlinx.coroutines +import kotlinx.coroutines.Runnable +import kotlinx.coroutines.scheduling.* +import kotlinx.coroutines.scheduling.CoroutineScheduler + internal actual abstract class EventLoopImplPlatform: EventLoop() { protected abstract val thread: Thread @@ -45,6 +49,80 @@ internal actual fun createEventLoop(): EventLoop = BlockingEventLoop(Thread.curr */ @InternalCoroutinesApi public fun processNextEventInCurrentThread(): Long = + // This API is used in Ktor for serverless integration where a single thread awaits a blocking call + // (and, to avoid actual blocking, does something via this call), see #850 ThreadLocalEventLoop.currentOrNull()?.processNextEvent() ?: Long.MAX_VALUE internal actual inline fun platformAutoreleasePool(crossinline block: () -> Unit) = block() + +/** + * Retrieves and executes a single task from the current system dispatcher ([Dispatchers.Default] or [Dispatchers.IO]). + * Returns `0` if any task was executed, `>= 0` for number of nanoseconds to wait until invoking this method again + * (implying that there will be a task to steal in N nanoseconds), `-1` if there is no tasks in the corresponding dispatcher at all. + * + * ### Invariants + * + * - When invoked from [Dispatchers.Default] **thread** (even if the actual context is different dispatcher, + * [CoroutineDispatcher.limitedParallelism] or any in-place wrapper), it runs an arbitrary task that ended + * up being scheduled to [Dispatchers.Default] or its counterpart. Tasks scheduled to [Dispatchers.IO] + * **are not** executed[1]. + * - When invoked from [Dispatchers.IO] thread, the same rules apply, but for blocking tasks only. + * + * [1] -- this is purely technical limitation: the scheduler does not have "notify me when CPU token is available" API, + * and we cannot leave this method without leaving thread in its original state. + * + * ### Rationale + * + * This is an internal API that is intended to replace IDEA's core FJP decomposition. + * The following API is provided by IDEA core: + * ``` + * runDecomposedTaskAndJoinIt { // <- non-suspending call + * // spawn as many tasks as needed + * // these tasks can also invoke 'runDecomposedTaskAndJoinIt' + * } + * ``` + * The key observation here is that 'runDecomposedTaskAndJoinIt' can be invoked from `Dispatchers.Default` itself, + * thus blocking at least one thread. To avoid deadlocks and starvation during large hierarchical decompositions, + * 'runDecomposedTaskAndJoinIt' should not just block but also **help** execute the task or other tasks + * until an arbitrary condition is satisfied. + * + * See #3439 for additional details. + * + * ### Limitations and caveats + * + * - Executes tasks in-place, thus potentially leaking irrelevant thread-locals from the current thread + * - Is not 100% effective, because the caller should somehow "wait" (or do other work) for [Long] returned nanoseconds + * even when work arrives immediately after returning from this method. + * - When there is no more work, it's up to the caller to decide what to do. It's important to remember that + * work to current dispatcher may arrive **later** from external sources [1] + * + * [1] -- this is also a technicality that can be solved in kotlinx.coroutines itself, but unfortunately requires + * a tremendous effort. + * + * @throws IllegalStateException if the current thread is not system dispatcher thread + */ +@InternalCoroutinesApi +@DelicateCoroutinesApi +@PublishedApi +internal fun runSingleTaskFromCurrentSystemDispatcher(): Long { + val thread = Thread.currentThread() + if (thread !is CoroutineScheduler.Worker) throw IllegalStateException("Expected CoroutineScheduler.Worker, but got $thread") + return thread.runSingleTask() +} + +/** + * Checks whether the given thread belongs to Dispatchers.IO. + * Note that feature "is part of the Dispatchers.IO" is *dynamic*, meaning that the thread + * may change this status when switching between tasks. + * + * This function is inteded to be used on the result of `Thread.currentThread()` for diagnostic + * purposes, and is declared as an extension only to avoid top-level scope pollution. + */ +@InternalCoroutinesApi +@DelicateCoroutinesApi +@PublishedApi +internal fun Thread.isIoDispatcherThread(): Boolean { + if (this !is CoroutineScheduler.Worker) return false + return isIo() +} + diff --git a/kotlinx-coroutines-core/jvm/src/Executors.kt b/kotlinx-coroutines-core/jvm/src/Executors.kt index 4e98e7bc..121ba3f4 100644 --- a/kotlinx-coroutines-core/jvm/src/Executors.kt +++ b/kotlinx-coroutines-core/jvm/src/Executors.kt @@ -108,7 +108,14 @@ public fun CoroutineDispatcher.asExecutor(): Executor = (this as? ExecutorCoroutineDispatcher)?.executor ?: DispatcherExecutor(this) private class DispatcherExecutor(@JvmField val dispatcher: CoroutineDispatcher) : Executor { - override fun execute(block: Runnable) = dispatcher.dispatch(EmptyCoroutineContext, block) + override fun execute(block: Runnable) { + if (dispatcher.isDispatchNeeded(EmptyCoroutineContext)) { + dispatcher.dispatch(EmptyCoroutineContext, block) + } else { + block.run() + } + } + override fun toString(): String = dispatcher.toString() } diff --git a/kotlinx-coroutines-core/jvm/src/Interruptible.kt b/kotlinx-coroutines-core/jvm/src/Interruptible.kt index 0bded765..8129be7b 100644 --- a/kotlinx-coroutines-core/jvm/src/Interruptible.kt +++ b/kotlinx-coroutines-core/jvm/src/Interruptible.kt @@ -1,5 +1,5 @@ /* - * Copyright 2016-2021 JetBrains s.r.o. Use of this source code is governed by the Apache 2.0 license. + * Copyright 2016-2023 JetBrains s.r.o. Use of this source code is governed by the Apache 2.0 license. */ package kotlinx.coroutines @@ -8,7 +8,8 @@ import kotlinx.atomicfu.* import kotlin.coroutines.* /** - * Calls the specified [block] with a given coroutine context in an interruptible manner. + * Calls the specified [block] with a given coroutine context in + * [an interruptible manner](https://docs.oracle.com/javase/tutorial/essential/concurrency/interrupt.html). * The blocking code block will be interrupted and this function will throw [CancellationException] * if the coroutine is cancelled. * diff --git a/kotlinx-coroutines-core/jvm/src/ThreadPoolDispatcher.kt b/kotlinx-coroutines-core/jvm/src/ThreadPoolDispatcher.kt index dc0b7e29..0828c0bc 100644 --- a/kotlinx-coroutines-core/jvm/src/ThreadPoolDispatcher.kt +++ b/kotlinx-coroutines-core/jvm/src/ThreadPoolDispatcher.kt @@ -2,39 +2,14 @@ * Copyright 2016-2021 JetBrains s.r.o. Use of this source code is governed by the Apache 2.0 license. */ +@file:JvmMultifileClass +@file:JvmName("ThreadPoolDispatcherKt") package kotlinx.coroutines import java.util.concurrent.* import java.util.concurrent.atomic.AtomicInteger /** - * Creates a coroutine execution context using a single thread with built-in [yield] support. - * **NOTE: The resulting [ExecutorCoroutineDispatcher] owns native resources (its thread). - * Resources are reclaimed by [ExecutorCoroutineDispatcher.close].** - * - * If the resulting dispatcher is [closed][ExecutorCoroutineDispatcher.close] and - * attempt to submit a continuation task is made, - * then the [Job] of the affected task is [cancelled][Job.cancel] and the task is submitted to the - * [Dispatchers.IO], so that the affected coroutine can cleanup its resources and promptly complete. - * - * This is a **delicate** API. The result of this method is a closeable resource with the - * associated native resources (threads). It should not be allocated in place, - * should be closed at the end of its lifecycle, and has non-trivial memory and CPU footprint. - * If you do not need a separate thread-pool, but only have to limit effective parallelism of the dispatcher, - * it is recommended to use [CoroutineDispatcher.limitedParallelism] instead. - * - * If you need a completely separate thread-pool with scheduling policy that is based on the standard - * JDK executors, use the following expression: - * `Executors.newSingleThreadExecutor().asCoroutineDispatcher()`. - * See [Executor.asCoroutineDispatcher] for details. - * - * @param name the base name of the created thread. - */ -@DelicateCoroutinesApi -public actual fun newSingleThreadContext(name: String): ExecutorCoroutineDispatcher = - newFixedThreadPoolContext(1, name) - -/** * Creates a coroutine execution context with the fixed-size thread-pool and built-in [yield] support. * **NOTE: The resulting [ExecutorCoroutineDispatcher] owns native resources (its threads). * Resources are reclaimed by [ExecutorCoroutineDispatcher.close].** diff --git a/kotlinx-coroutines-core/jvm/src/channels/Actor.kt b/kotlinx-coroutines-core/jvm/src/channels/Actor.kt index 748f5283..e8a9152e 100644 --- a/kotlinx-coroutines-core/jvm/src/channels/Actor.kt +++ b/kotlinx-coroutines-core/jvm/src/channels/Actor.kt @@ -149,8 +149,7 @@ private class LazyActorCoroutine<E>( parentContext: CoroutineContext, channel: Channel<E>, block: suspend ActorScope<E>.() -> Unit -) : ActorCoroutine<E>(parentContext, channel, active = false), - SelectClause2<E, SendChannel<E>> { +) : ActorCoroutine<E>(parentContext, channel, active = false) { private var continuation = block.createCoroutineUnintercepted(this, this) @@ -163,7 +162,12 @@ private class LazyActorCoroutine<E>( return super.send(element) } - @Suppress("DEPRECATION", "DEPRECATION_ERROR") + @Suppress("DEPRECATION_ERROR") + @Deprecated( + level = DeprecationLevel.ERROR, + message = "Deprecated in the favour of 'trySend' method", + replaceWith = ReplaceWith("trySend(element).isSuccess") + ) // See super() override fun offer(element: E): Boolean { start() return super.offer(element) @@ -182,12 +186,15 @@ private class LazyActorCoroutine<E>( return closed } - override val onSend: SelectClause2<E, SendChannel<E>> - get() = this + @Suppress("UNCHECKED_CAST") + override val onSend: SelectClause2<E, SendChannel<E>> get() = SelectClause2Impl( + clauseObject = this, + regFunc = LazyActorCoroutine<*>::onSendRegFunction as RegistrationFunction, + processResFunc = super.onSend.processResFunc + ) - // registerSelectSend - override fun <R> registerSelectClause2(select: SelectInstance<R>, param: E, block: suspend (SendChannel<E>) -> R) { - start() - super.onSend.registerSelectClause2(select, param, block) + private fun onSendRegFunction(select: SelectInstance<*>, element: Any?) { + onStart() + super.onSend.regFunc(this, select, element) } } diff --git a/kotlinx-coroutines-core/jvm/src/debug/AgentPremain.kt b/kotlinx-coroutines-core/jvm/src/debug/AgentPremain.kt index 4b0ce3f3..fb5c5b1b 100644 --- a/kotlinx-coroutines-core/jvm/src/debug/AgentPremain.kt +++ b/kotlinx-coroutines-core/jvm/src/debug/AgentPremain.kt @@ -26,6 +26,7 @@ internal object AgentPremain { }.getOrNull() ?: DebugProbesImpl.enableCreationStackTraces @JvmStatic + @Suppress("UNUSED_PARAMETER") fun premain(args: String?, instrumentation: Instrumentation) { AgentInstallationType.isInstalledStatically = true instrumentation.addTransformer(DebugProbesTransformer) @@ -36,13 +37,13 @@ internal object AgentPremain { internal object DebugProbesTransformer : ClassFileTransformer { override fun transform( - loader: ClassLoader, + loader: ClassLoader?, className: String, classBeingRedefined: Class<*>?, protectionDomain: ProtectionDomain, classfileBuffer: ByteArray? ): ByteArray? { - if (className != "kotlin/coroutines/jvm/internal/DebugProbesKt") { + if (loader == null || className != "kotlin/coroutines/jvm/internal/DebugProbesKt") { return null } /* diff --git a/kotlinx-coroutines-core/jvm/src/debug/CoroutineDebugging.kt b/kotlinx-coroutines-core/jvm/src/debug/CoroutineDebugging.kt new file mode 100644 index 00000000..49a794e0 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/src/debug/CoroutineDebugging.kt @@ -0,0 +1,65 @@ +/* + * Copyright 2016-2020 JetBrains s.r.o. Use of this source code is governed by the Apache 2.0 license. + */ + +/* This package name is like this so that +1) the artificial stack frames look pretty, and +2) the IDE reliably navigates to this file. */ +package _COROUTINE + +/** + * A collection of artificial stack trace elements to be included in stack traces by the coroutines machinery. + * + * There are typically two ways in which one can encounter an artificial stack frame: + * 1. By using the debug mode, via the stacktrace recovery mechanism; see + * [stacktrace recovery](https://github.com/Kotlin/kotlinx.coroutines/blob/master/docs/topics/debugging.md#stacktrace-recovery) + * documentation. The usual way to enable the debug mode is with the [kotlinx.coroutines.DEBUG_PROPERTY_NAME] system + * property. + * 2. By looking at the output of DebugProbes; see the + * [kotlinx-coroutines-debug](https://github.com/Kotlin/kotlinx.coroutines/tree/master/kotlinx-coroutines-debug) module. + */ +internal class ArtificialStackFrames { + /** + * Returns an artificial stack trace element denoting the boundary between coroutine creation and its execution. + * + * Appearance of this function in stack traces does not mean that it was called. Instead, it is used as a marker + * that separates the part of the stack trace with the code executed in a coroutine from the stack trace of the code + * that launched the coroutine. + * + * In earlier versions of kotlinx-coroutines, this was displayed as "(Coroutine creation stacktrace)", which caused + * problems for tooling that processes stack traces: https://github.com/Kotlin/kotlinx.coroutines/issues/2291 + * + * Note that presence of this marker in a stack trace implies that coroutine creation stack traces were enabled. + */ + fun coroutineCreation(): StackTraceElement = Exception().artificialFrame(_CREATION::class.java.simpleName) + + /** + * Returns an artificial stack trace element denoting a coroutine boundary. + * + * Appearance of this function in stack traces does not mean that it was called. Instead, when one coroutine invokes + * another, this is used as a marker in the stack trace to denote where the execution of one coroutine ends and that + * of another begins. + * + * In earlier versions of kotlinx-coroutines, this was displayed as "(Coroutine boundary)", which caused + * problems for tooling that processes stack traces: https://github.com/Kotlin/kotlinx.coroutines/issues/2291 + */ + fun coroutineBoundary(): StackTraceElement = Exception().artificialFrame(_BOUNDARY::class.java.simpleName) +} + +// These are needed for the IDE navigation to detect that this file does contain the definition. +private class _CREATION +private class _BOUNDARY + +internal val ARTIFICIAL_FRAME_PACKAGE_NAME = "_COROUTINE" + +/** + * Forms an artificial stack frame with the given class name. + * + * It consists of the following parts: + * 1. The package name, it seems, is needed for the IDE to detect stack trace elements reliably. It is `_COROUTINE` since + * this is a valid identifier. + * 2. Class names represents what type of artificial frame this is. + * 3. The method name is `_`. The methods not being present in class definitions does not seem to affect navigation. + */ +private fun Throwable.artificialFrame(name: String): StackTraceElement = + with(stackTrace[0]) { StackTraceElement(ARTIFICIAL_FRAME_PACKAGE_NAME + "." + name, "_", fileName, lineNumber) } diff --git a/kotlinx-coroutines-core/jvm/src/debug/internal/DebugCoroutineInfo.kt b/kotlinx-coroutines-core/jvm/src/debug/internal/DebugCoroutineInfo.kt index 6c353929..56ca6c51 100644 --- a/kotlinx-coroutines-core/jvm/src/debug/internal/DebugCoroutineInfo.kt +++ b/kotlinx-coroutines-core/jvm/src/debug/internal/DebugCoroutineInfo.kt @@ -15,11 +15,11 @@ import kotlin.coroutines.jvm.internal.* */ @Suppress("unused") @PublishedApi -internal class DebugCoroutineInfo( +internal class DebugCoroutineInfo internal constructor( source: DebugCoroutineInfoImpl, public val context: CoroutineContext // field is used as of 1.4-M3 ) { - public val creationStackBottom: CoroutineStackFrame? = source.creationStackBottom // field is used as of 1.4-M3 + internal val creationStackBottom: CoroutineStackFrame? = source.creationStackBottom // field is used as of 1.4-M3 public val sequenceNumber: Long = source.sequenceNumber // field is used as of 1.4-M3 public val creationStackTrace = source.creationStackTrace // getter is used as of 1.4-M3 public val state: String = source.state // getter is used as of 1.4-M3 diff --git a/kotlinx-coroutines-core/jvm/src/debug/internal/DebugCoroutineInfoImpl.kt b/kotlinx-coroutines-core/jvm/src/debug/internal/DebugCoroutineInfoImpl.kt index 07b9419f..261d247c 100644 --- a/kotlinx-coroutines-core/jvm/src/debug/internal/DebugCoroutineInfoImpl.kt +++ b/kotlinx-coroutines-core/jvm/src/debug/internal/DebugCoroutineInfoImpl.kt @@ -14,15 +14,18 @@ internal const val SUSPENDED = "SUSPENDED" /** * Internal implementation class where debugger tracks details it knows about each coroutine. + * Its mutable fields can be updated concurrently, thus marked with `@Volatile` */ -internal class DebugCoroutineInfoImpl( +@PublishedApi +internal class DebugCoroutineInfoImpl internal constructor( context: CoroutineContext?, /** * A reference to a stack-trace that is converted to a [StackTraceFrame] which implements [CoroutineStackFrame]. * The actual reference to the coroutine is not stored here, so we keep a strong reference. */ - public val creationStackBottom: StackTraceFrame?, - @JvmField internal val sequenceNumber: Long + internal val creationStackBottom: StackTraceFrame?, + // Used by the IDEA debugger via reflection and must be kept binary-compatible, see KTIJ-24102 + @JvmField public val sequenceNumber: Long ) { /** * We cannot keep a strong reference to the context, because with the [Job] in the context it will indirectly @@ -39,27 +42,115 @@ internal class DebugCoroutineInfoImpl( * Last observed state of the coroutine. * Can be CREATED, RUNNING, SUSPENDED. */ - public val state: String get() = _state - private var _state: String = CREATED + internal val state: String get() = _state + // Used by the IDEA debugger via reflection and must be kept binary-compatible, see KTIJ-24102 + @Volatile @JvmField - internal var lastObservedThread: Thread? = null + public var _state: String = CREATED + + /* + * How many consecutive unmatched 'updateState(RESUMED)' this object has received. + * It can be `> 1` in two cases: + * + * * The coroutine is finishing and its state is being unrolled in BaseContinuationImpl, see comment to DebugProbesImpl#callerInfoCache + * Such resumes are not expected to be matched and are ignored. + * * We encountered suspend-resume race explained above, and we do wait for a match. + */ + private var unmatchedResume = 0 + + /** + * Here we orchestrate overlapping state updates that are coming asynchronously. + * In a nutshell, `probeCoroutineSuspended` can arrive **later** than its matching `probeCoroutineResumed`, + * e.g. for the following code: + * ``` + * suspend fun foo() = yield() + * ``` + * + * we have this sequence: + * ``` + * fun foo(...) { + * uCont.intercepted().dispatchUsingDispatcher() // 1 + * // Notify the debugger the coroutine is suspended + * probeCoroutineSuspended() // 2 + * return COROUTINE_SUSPENDED // Unroll the stack + * } + * ``` + * Nothing prevents coroutine to be dispatched and invoke `probeCoroutineResumed` right between '1' and '2'. + * See also: https://github.com/Kotlin/kotlinx.coroutines/issues/3193 + * + * [shouldBeMatched] -- `false` if it is an expected consecutive `probeCoroutineResumed` from BaseContinuationImpl, + * `true` otherwise. + */ + @Synchronized + internal fun updateState(state: String, frame: Continuation<*>, shouldBeMatched: Boolean) { + /** + * We observe consecutive resume that had to be matched, but it wasn't, + * increment + */ + if (_state == RUNNING && state == RUNNING && shouldBeMatched) { + ++unmatchedResume + } else if (unmatchedResume > 0 && state == SUSPENDED) { + /* + * We received late 'suspend' probe for unmatched resume, skip it. + * Here we deliberately allow the very unlikely race; + * Consider the following scenario ('[r:a]' means "probeCoroutineResumed at a()"): + * ``` + * [r:a] a() -> b() [s:b] [r:b] -> (back to a) a() -> c() [s:c] + * ``` + * We can, in theory, observe the following probes interleaving: + * ``` + * r:a + * r:b // Unmatched resume + * s:c // Matched suspend, discard + * s:b + * ``` + * Thus mis-attributing 'lastObservedFrame' to a previously-observed. + * It is possible in theory (though I've failed to reproduce it), yet + * is more preferred than indefinitely mismatched state (-> mismatched real/enhanced stacktrace) + */ + --unmatchedResume + return + } + + // Propagate only non-duplicating transitions to running, see KT-29997 + if (_state == state && state == SUSPENDED && lastObservedFrame != null) return + + _state = state + lastObservedFrame = frame as? CoroutineStackFrame + lastObservedThread = if (state == RUNNING) { + Thread.currentThread() + } else { + null + } + } + + // Used by the IDEA debugger via reflection and must be kept binary-compatible, see KTIJ-24102 + @JvmField + @Volatile + public var lastObservedThread: Thread? = null /** * We cannot keep a strong reference to the last observed frame of the coroutine, because this will * prevent garbage-collection of a coroutine that was lost. + * + * Used by the IDEA debugger via reflection and must be kept binary-compatible, see KTIJ-24102 */ - private var _lastObservedFrame: WeakReference<CoroutineStackFrame>? = null + @Volatile + @JvmField + public var _lastObservedFrame: WeakReference<CoroutineStackFrame>? = null internal var lastObservedFrame: CoroutineStackFrame? get() = _lastObservedFrame?.get() - set(value) { _lastObservedFrame = value?.let { WeakReference(it) } } + set(value) { + _lastObservedFrame = value?.let { WeakReference(it) } + } /** * Last observed stacktrace of the coroutine captured on its suspension or resumption point. * It means that for [running][State.RUNNING] coroutines resulting stacktrace is inaccurate and * reflects stacktrace of the resumption point, not the actual current stacktrace. */ - public fun lastObservedStackTrace(): List<StackTraceElement> { + internal fun lastObservedStackTrace(): List<StackTraceElement> { var frame: CoroutineStackFrame? = lastObservedFrame ?: return emptyList() val result = ArrayList<StackTraceElement>() while (frame != null) { @@ -84,17 +175,5 @@ internal class DebugCoroutineInfoImpl( } } - internal fun updateState(state: String, frame: Continuation<*>) { - // Propagate only duplicating transitions to running for KT-29997 - if (_state == state && state == SUSPENDED && lastObservedFrame != null) return - _state = state - lastObservedFrame = frame as? CoroutineStackFrame - lastObservedThread = if (state == RUNNING) { - Thread.currentThread() - } else { - null - } - } - override fun toString(): String = "DebugCoroutineInfo(state=$state,context=$context)" } diff --git a/kotlinx-coroutines-core/jvm/src/debug/internal/DebugProbesImpl.kt b/kotlinx-coroutines-core/jvm/src/debug/internal/DebugProbesImpl.kt index d358d49d..c11028c0 100644 --- a/kotlinx-coroutines-core/jvm/src/debug/internal/DebugProbesImpl.kt +++ b/kotlinx-coroutines-core/jvm/src/debug/internal/DebugProbesImpl.kt @@ -6,7 +6,6 @@ package kotlinx.coroutines.debug.internal import kotlinx.atomicfu.* import kotlinx.coroutines.* -import kotlinx.coroutines.internal.* import kotlinx.coroutines.internal.ScopeCoroutine import java.io.* import java.lang.StackTraceElement @@ -17,10 +16,11 @@ import kotlin.concurrent.* import kotlin.coroutines.* import kotlin.coroutines.jvm.internal.CoroutineStackFrame import kotlin.synchronized -import kotlinx.coroutines.internal.artificialFrame as createArtificialFrame // IDEA bug workaround +import _COROUTINE.ArtificialStackFrames +@PublishedApi internal object DebugProbesImpl { - private const val ARTIFICIAL_FRAME_MESSAGE = "Coroutine creation stacktrace" + private val ARTIFICIAL_FRAME = ArtificialStackFrames().coroutineCreation() private val dateFormat = SimpleDateFormat("yyyy/MM/dd HH:mm:ss") private var weakRefCleanerThread: Thread? = null @@ -29,32 +29,24 @@ internal object DebugProbesImpl { private val capturedCoroutinesMap = ConcurrentWeakMap<CoroutineOwner<*>, Boolean>() private val capturedCoroutines: Set<CoroutineOwner<*>> get() = capturedCoroutinesMap.keys - @Volatile - private var installations = 0 + private val installations = atomic(0) /** - * This internal method is used by IDEA debugger under the JVM name of - * "isInstalled$kotlinx_coroutines_debug". + * This internal method is used by the IDEA debugger under the JVM name + * "isInstalled$kotlinx_coroutines_debug" and must be kept binary-compatible, see KTIJ-24102 */ - internal val isInstalled: Boolean get() = installations > 0 + val isInstalled: Boolean get() = installations.value > 0 - // To sort coroutines by creation order, used as unique id + // To sort coroutines by creation order, used as a unique id private val sequenceNumber = atomic(0L) - /* - * RW-lock that guards all debug probes state changes. - * All individual coroutine state transitions are guarded by read-lock - * and do not interfere with each other. - * All state reads are guarded by the write lock to guarantee a strongly-consistent - * snapshot of the system. - */ - private val coroutineStateLock = ReentrantReadWriteLock() - public var sanitizeStackTraces: Boolean = true - public var enableCreationStackTraces: Boolean = true + internal var sanitizeStackTraces: Boolean = true + internal var enableCreationStackTraces: Boolean = true + public var ignoreCoroutinesWithEmptyContext: Boolean = true /* * Substitute for service loader, DI between core and debug modules. - * If the agent was installed via command line -javaagent parameter, do not use byte-byddy to avoud + * If the agent was installed via command line -javaagent parameter, do not use byte-buddy to avoid dynamic attach. */ private val dynamicAttach = getDynamicAttach() @@ -65,29 +57,30 @@ internal object DebugProbesImpl { ctor.newInstance() as Function1<Boolean, Unit> }.getOrNull() - /* - * This is an optimization in the face of KT-29997: - * Consider suspending call stack a()->b()->c() and c() completes its execution and every call is + /** + * Because `probeCoroutinesResumed` is called for every resumed continuation (see KT-29997 and the related code), + * we perform a performance optimization: + * Imagine a suspending call stack a()->b()->c(), where c() completes its execution and every call is * "almost" in tail position. * - * Then at least three RUNNING -> RUNNING transitions will occur consecutively and complexity of each is O(depth). - * To avoid that quadratic complexity, we are caching lookup result for such chains in this map and update it incrementally. + * Then at least three RUNNING -> RUNNING transitions will occur consecutively, the complexity of each O(depth). + * To avoid this quadratic complexity, we are caching lookup result for such chains in this map and update it incrementally. * * [DebugCoroutineInfoImpl] keeps a lot of auxiliary information about a coroutine, so we use a weak reference queue * to promptly release the corresponding memory when the reference to the coroutine itself was already collected. */ private val callerInfoCache = ConcurrentWeakMap<CoroutineStackFrame, DebugCoroutineInfoImpl>(weakRefQueue = true) - public fun install(): Unit = coroutineStateLock.write { - if (++installations > 1) return + internal fun install() { + if (installations.incrementAndGet() > 1) return startWeakRefCleanerThread() if (AgentInstallationType.isInstalledStatically) return dynamicAttach?.invoke(true) // attach } - public fun uninstall(): Unit = coroutineStateLock.write { + internal fun uninstall() { check(isInstalled) { "Agent was not installed" } - if (--installations != 0) return + if (installations.decrementAndGet() != 0) return stopWeakRefCleanerThread() capturedCoroutinesMap.clear() callerInfoCache.clear() @@ -108,7 +101,7 @@ internal object DebugProbesImpl { thread.join() } - public fun hierarchyToString(job: Job): String = coroutineStateLock.write { + internal fun hierarchyToString(job: Job): String { check(isInstalled) { "Debug probes are not installed" } val jobToStack = capturedCoroutines .filter { it.delegate.context[Job] != null } @@ -150,20 +143,19 @@ internal object DebugProbesImpl { * Private method that dumps coroutines so that different public-facing method can use * to produce different result types. */ - private inline fun <R : Any> dumpCoroutinesInfoImpl(crossinline create: (CoroutineOwner<*>, CoroutineContext) -> R): List<R> = - coroutineStateLock.write { - check(isInstalled) { "Debug probes are not installed" } - capturedCoroutines - .asSequence() - // Stable ordering of coroutines by their sequence number - .sortedBy { it.info.sequenceNumber } - // Leave in the dump only the coroutines that were not collected while we were dumping them - .mapNotNull { owner -> - // Fuse map and filter into one operation to save an inline - if (owner.isFinished()) null - else owner.info.context?.let { context -> create(owner, context) } - }.toList() - } + private inline fun <R : Any> dumpCoroutinesInfoImpl(crossinline create: (CoroutineOwner<*>, CoroutineContext) -> R): List<R> { + check(isInstalled) { "Debug probes are not installed" } + return capturedCoroutines + .asSequence() + // Stable ordering of coroutines by their sequence number + .sortedBy { it.info.sequenceNumber } + // Leave in the dump only the coroutines that were not collected while we were dumping them + .mapNotNull { owner -> + // Fuse map and filter into one operation to save an inline + if (owner.isFinished()) null + else owner.info.context?.let { context -> create(owner, context) } + }.toList() + } /* * This method optimises the number of packages sent by the IDEA debugger @@ -183,9 +175,10 @@ internal object DebugProbesImpl { * to save an exponential number of roundtrips. * * Internal (JVM-public) method used by IDEA debugger as of 1.6.0-RC. + * See KTIJ-24102. */ @OptIn(ExperimentalStdlibApi::class) - public fun dumpCoroutinesInfoAsJsonAndReferences(): Array<Any> { + fun dumpCoroutinesInfoAsJsonAndReferences(): Array<Any> { val coroutinesInfo = dumpCoroutinesInfo() val size = coroutinesInfo.size val lastObservedThreads = ArrayList<Thread?>(size) @@ -193,8 +186,8 @@ internal object DebugProbesImpl { val coroutinesInfoAsJson = ArrayList<String>(size) for (info in coroutinesInfo) { val context = info.context - val name = context[CoroutineName.Key]?.name?.toStringWithQuotes() - val dispatcher = context[CoroutineDispatcher.Key]?.toStringWithQuotes() + val name = context[CoroutineName.Key]?.name?.toStringRepr() + val dispatcher = context[CoroutineDispatcher.Key]?.toStringRepr() coroutinesInfoAsJson.add( """ { @@ -219,9 +212,9 @@ internal object DebugProbesImpl { } /* - * Internal (JVM-public) method used by IDEA debugger as of 1.6.0-RC. + * Internal (JVM-public) method used by IDEA debugger as of 1.6.0-RC, must be kept binary-compatible, see KTIJ-24102 */ - public fun enhanceStackTraceWithThreadDumpAsJson(info: DebugCoroutineInfo): String { + fun enhanceStackTraceWithThreadDumpAsJson(info: DebugCoroutineInfo): String { val stackTraceElements = enhanceStackTraceWithThreadDump(info, info.lastObservedStackTrace) val stackTraceElementsInfoAsJson = mutableListOf<String>() for (element in stackTraceElements) { @@ -230,7 +223,7 @@ internal object DebugProbesImpl { { "declaringClass": "${element.className}", "methodName": "${element.methodName}", - "fileName": ${element.fileName?.toStringWithQuotes()}, + "fileName": ${element.fileName?.toStringRepr()}, "lineNumber": ${element.lineNumber} } """.trimIndent() @@ -240,22 +233,23 @@ internal object DebugProbesImpl { return "[${stackTraceElementsInfoAsJson.joinToString()}]" } - private fun Any.toStringWithQuotes() = "\"$this\"" + private fun Any.toStringRepr() = toString().repr() /* - * Internal (JVM-public) method used by IDEA debugger as of 1.4-M3. + * Internal (JVM-public) method used by IDEA debugger as of 1.4-M3. See KTIJ-24102 */ - public fun dumpCoroutinesInfo(): List<DebugCoroutineInfo> = + fun dumpCoroutinesInfo(): List<DebugCoroutineInfo> = dumpCoroutinesInfoImpl { owner, context -> DebugCoroutineInfo(owner.info, context) } /* * Internal (JVM-public) method to be used by IDEA debugger in the future (not used as of 1.4-M3). * It is equivalent to [dumpCoroutinesInfo], but returns serializable (and thus less typed) objects. */ - public fun dumpDebuggerInfo(): List<DebuggerInfo> = + fun dumpDebuggerInfo(): List<DebuggerInfo> = dumpCoroutinesInfoImpl { owner, context -> DebuggerInfo(owner.info, context) } - public fun dumpCoroutines(out: PrintStream): Unit = synchronized(out) { + @JvmName("dumpCoroutines") + internal fun dumpCoroutines(out: PrintStream): Unit = synchronized(out) { /* * This method synchronizes both on `out` and `this` for a reason: * 1) Taking a write lock is required to have a consistent snapshot of coroutines. @@ -281,7 +275,7 @@ internal object DebugProbesImpl { return true } - private fun dumpCoroutinesSynchronized(out: PrintStream): Unit = coroutineStateLock.write { + private fun dumpCoroutinesSynchronized(out: PrintStream) { check(isInstalled) { "Debug probes are not installed" } out.print("Coroutines dump ${dateFormat.format(System.currentTimeMillis())}") capturedCoroutines @@ -298,7 +292,7 @@ internal object DebugProbesImpl { info.state out.print("\n\nCoroutine ${owner.delegate}, state: $state") if (observedStackTrace.isEmpty()) { - out.print("\n\tat ${createArtificialFrame(ARTIFICIAL_FRAME_MESSAGE)}") + out.print("\n\tat $ARTIFICIAL_FRAME") printStackTrace(out, info.creationStackTrace) } else { printStackTrace(out, enhancedStackTrace) @@ -313,11 +307,11 @@ internal object DebugProbesImpl { } /* - * Internal (JVM-public) method used by IDEA debugger as of 1.4-M3. + * Internal (JVM-public) method used by IDEA debugger as of 1.4-M3, must be kept binary-compatible. See KTIJ-24102. * It is similar to [enhanceStackTraceWithThreadDumpImpl], but uses debugger-facing [DebugCoroutineInfo] type. */ @Suppress("unused") - public fun enhanceStackTraceWithThreadDump( + fun enhanceStackTraceWithThreadDump( info: DebugCoroutineInfo, coroutineTrace: List<StackTraceElement> ): List<StackTraceElement> = @@ -429,8 +423,8 @@ internal object DebugProbesImpl { private fun updateState(frame: Continuation<*>, state: String) { if (!isInstalled) return - // KT-29997 is here only since 1.3.30 - if (state == RUNNING && KotlinVersion.CURRENT.isAtLeast(1, 3, 30)) { + if (ignoreCoroutinesWithEmptyContext && frame.context === EmptyCoroutineContext) return // See ignoreCoroutinesWithEmptyContext + if (state == RUNNING) { val stackFrame = frame as? CoroutineStackFrame ?: return updateRunningState(stackFrame, state) return @@ -442,21 +436,23 @@ internal object DebugProbesImpl { } // See comment to callerInfoCache - private fun updateRunningState(frame: CoroutineStackFrame, state: String): Unit = coroutineStateLock.read { + private fun updateRunningState(frame: CoroutineStackFrame, state: String) { if (!isInstalled) return // Lookup coroutine info in cache or by traversing stack frame val info: DebugCoroutineInfoImpl val cached = callerInfoCache.remove(frame) + val shouldBeMatchedWithProbeSuspended: Boolean if (cached != null) { info = cached + shouldBeMatchedWithProbeSuspended = false } else { info = frame.owner()?.info ?: return + shouldBeMatchedWithProbeSuspended = true // Guard against improper implementations of CoroutineStackFrame and bugs in the compiler val realCaller = info.lastObservedFrame?.realCaller() if (realCaller != null) callerInfoCache.remove(realCaller) } - - info.updateState(state, frame as Continuation<*>) + info.updateState(state, frame as Continuation<*>, shouldBeMatchedWithProbeSuspended) // Do not cache it for proxy-classes such as ScopeCoroutines val caller = frame.realCaller() ?: return callerInfoCache[caller] = info @@ -467,9 +463,9 @@ internal object DebugProbesImpl { return if (caller.getStackTraceElement() != null) caller else caller.realCaller() } - private fun updateState(owner: CoroutineOwner<*>, frame: Continuation<*>, state: String) = coroutineStateLock.read { + private fun updateState(owner: CoroutineOwner<*>, frame: Continuation<*>, state: String) { if (!isInstalled) return - owner.info.updateState(state, frame) + owner.info.updateState(state, frame, true) } private fun Continuation<*>.owner(): CoroutineOwner<*>? = (this as? CoroutineStackFrame)?.owner() @@ -480,6 +476,8 @@ internal object DebugProbesImpl { // Not guarded by the lock at all, does not really affect consistency internal fun <T> probeCoroutineCreated(completion: Continuation<T>): Continuation<T> { if (!isInstalled) return completion + // See DebugProbes.ignoreCoroutinesWithEmptyContext for the additional details. + if (ignoreCoroutinesWithEmptyContext && completion.context === EmptyCoroutineContext) return completion /* * If completion already has an owner, it means that we are in scoped coroutine (coroutineScope, withContext etc.), * then piggyback on its already existing owner and do not replace completion @@ -500,15 +498,17 @@ internal object DebugProbesImpl { return createOwner(completion, frame) } - private fun List<StackTraceElement>.toStackTraceFrame(): StackTraceFrame? = - foldRight<StackTraceElement, StackTraceFrame?>(null) { frame, acc -> - StackTraceFrame(acc, frame) - } + private fun List<StackTraceElement>.toStackTraceFrame(): StackTraceFrame = + StackTraceFrame( + foldRight<StackTraceElement, StackTraceFrame?>(null) { frame, acc -> + StackTraceFrame(acc, frame) + }, ARTIFICIAL_FRAME + ) private fun <T> createOwner(completion: Continuation<T>, frame: StackTraceFrame?): Continuation<T> { if (!isInstalled) return completion val info = DebugCoroutineInfoImpl(completion.context, frame, sequenceNumber.incrementAndGet()) - val owner = CoroutineOwner(completion, info, frame) + val owner = CoroutineOwner(completion, info) capturedCoroutinesMap[owner] = true if (!isInstalled) capturedCoroutinesMap.clear() return owner @@ -529,11 +529,12 @@ internal object DebugProbesImpl { * This class is injected as completion of all continuations in [probeCoroutineCompleted]. * It is owning the coroutine info and responsible for managing all its external info related to debug agent. */ - private class CoroutineOwner<T>( - @JvmField val delegate: Continuation<T>, - @JvmField val info: DebugCoroutineInfoImpl, - private val frame: CoroutineStackFrame? + public class CoroutineOwner<T> internal constructor( + @JvmField internal val delegate: Continuation<T>, + // Used by the IDEA debugger via reflection and must be kept binary-compatible, see KTIJ-24102 + @JvmField public val info: DebugCoroutineInfoImpl ) : Continuation<T> by delegate, CoroutineStackFrame { + private val frame get() = info.creationStackBottom override val callerFrame: CoroutineStackFrame? get() = frame?.callerFrame @@ -551,12 +552,10 @@ internal object DebugProbesImpl { private fun <T : Throwable> sanitizeStackTrace(throwable: T): List<StackTraceElement> { val stackTrace = throwable.stackTrace val size = stackTrace.size - val probeIndex = stackTrace.indexOfLast { it.className == "kotlin.coroutines.jvm.internal.DebugProbesKt" } + val traceStart = 1 + stackTrace.indexOfLast { it.className == "kotlin.coroutines.jvm.internal.DebugProbesKt" } if (!sanitizeStackTraces) { - return List(size - probeIndex) { - if (it == 0) createArtificialFrame(ARTIFICIAL_FRAME_MESSAGE) else stackTrace[it + probeIndex] - } + return List(size - traceStart) { stackTrace[it + traceStart] } } /* @@ -567,9 +566,8 @@ internal object DebugProbesImpl { * If an interval of internal methods ends in a synthetic method, the outermost non-synthetic method in that * interval will also be included. */ - val result = ArrayList<StackTraceElement>(size - probeIndex + 1) - result += createArtificialFrame(ARTIFICIAL_FRAME_MESSAGE) - var i = probeIndex + 1 + val result = ArrayList<StackTraceElement>(size - traceStart + 1) + var i = traceStart while (i < size) { if (stackTrace[i].isInternalMethod) { result += stackTrace[i] // we include the boundary of the span in any case @@ -600,3 +598,19 @@ internal object DebugProbesImpl { private val StackTraceElement.isInternalMethod: Boolean get() = className.startsWith("kotlinx.coroutines") } + +private fun String.repr(): String = buildString { + append('"') + for (c in this@repr) { + when (c) { + '"' -> append("\\\"") + '\\' -> append("\\\\") + '\b' -> append("\\b") + '\n' -> append("\\n") + '\r' -> append("\\r") + '\t' -> append("\\t") + else -> append(c) + } + } + append('"') +} diff --git a/kotlinx-coroutines-core/jvm/src/debug/internal/StackTraceFrame.kt b/kotlinx-coroutines-core/jvm/src/debug/internal/StackTraceFrame.kt index 40101192..6cdd5f21 100644 --- a/kotlinx-coroutines-core/jvm/src/debug/internal/StackTraceFrame.kt +++ b/kotlinx-coroutines-core/jvm/src/debug/internal/StackTraceFrame.kt @@ -9,9 +9,11 @@ import kotlin.coroutines.jvm.internal.* /** * A stack-trace represented as [CoroutineStackFrame]. */ -internal class StackTraceFrame( +@PublishedApi +internal class StackTraceFrame internal constructor( override val callerFrame: CoroutineStackFrame?, - private val stackTraceElement: StackTraceElement + // Used by the IDEA debugger via reflection and must be kept binary-compatible, see KTIJ-24102 + @JvmField public val stackTraceElement: StackTraceElement ) : CoroutineStackFrame { override fun getStackTraceElement(): StackTraceElement = stackTraceElement } diff --git a/kotlinx-coroutines-core/jvm/src/internal/Concurrent.kt b/kotlinx-coroutines-core/jvm/src/internal/Concurrent.kt index 050b9747..5df79b8d 100644 --- a/kotlinx-coroutines-core/jvm/src/internal/Concurrent.kt +++ b/kotlinx-coroutines-core/jvm/src/internal/Concurrent.kt @@ -9,8 +9,6 @@ import java.util.* import java.util.concurrent.* import kotlin.concurrent.withLock as withLockJvm -internal actual fun <E> subscriberList(): SubscribersList<E> = CopyOnWriteArrayList() - @Suppress("ACTUAL_WITHOUT_EXPECT") internal actual typealias ReentrantLock = java.util.concurrent.locks.ReentrantLock diff --git a/kotlinx-coroutines-core/jvm/src/internal/CoroutineExceptionHandlerImpl.kt b/kotlinx-coroutines-core/jvm/src/internal/CoroutineExceptionHandlerImpl.kt new file mode 100644 index 00000000..7f11898a --- /dev/null +++ b/kotlinx-coroutines-core/jvm/src/internal/CoroutineExceptionHandlerImpl.kt @@ -0,0 +1,49 @@ +/* + * Copyright 2016-2022 JetBrains s.r.o. Use of this source code is governed by the Apache 2.0 license. + */ + +package kotlinx.coroutines.internal + +import java.util.* +import kotlinx.coroutines.* +import kotlin.coroutines.* + +/** + * A list of globally installed [CoroutineExceptionHandler] instances. + * + * Note that Android may have dummy [Thread.contextClassLoader] which is used by one-argument [ServiceLoader.load] function, + * see (https://stackoverflow.com/questions/13407006/android-class-loader-may-fail-for-processes-that-host-multiple-applications). + * So here we explicitly use two-argument `load` with a class-loader of [CoroutineExceptionHandler] class. + * + * We are explicitly using the `ServiceLoader.load(MyClass::class.java, MyClass::class.java.classLoader).iterator()` + * form of the ServiceLoader call to enable R8 optimization when compiled on Android. + */ +internal actual val platformExceptionHandlers: Collection<CoroutineExceptionHandler> = ServiceLoader.load( + CoroutineExceptionHandler::class.java, + CoroutineExceptionHandler::class.java.classLoader +).iterator().asSequence().toList() + +internal actual fun ensurePlatformExceptionHandlerLoaded(callback: CoroutineExceptionHandler) { + // we use JVM's mechanism of ServiceLoader, so this should be a no-op on JVM. + // The only thing we do is make sure that the ServiceLoader did work correctly. + check(callback in platformExceptionHandlers) { "Exception handler was not found via a ServiceLoader" } +} + +internal actual fun propagateExceptionFinalResort(exception: Throwable) { + // use the thread's handler + val currentThread = Thread.currentThread() + currentThread.uncaughtExceptionHandler.uncaughtException(currentThread, exception) +} + +// This implementation doesn't store a stacktrace, which is good because a stacktrace doesn't make sense for this. +internal actual class DiagnosticCoroutineContextException actual constructor(@Transient private val context: CoroutineContext) : RuntimeException() { + override fun getLocalizedMessage(): String { + return context.toString() + } + + override fun fillInStackTrace(): Throwable { + // Prevent Android <= 6.0 bug, #1866 + stackTrace = emptyArray() + return this + } +} diff --git a/kotlinx-coroutines-core/jvm/src/internal/ExceptionsConstructor.kt b/kotlinx-coroutines-core/jvm/src/internal/ExceptionsConstructor.kt index a4b321d8..4f425a31 100644 --- a/kotlinx-coroutines-core/jvm/src/internal/ExceptionsConstructor.kt +++ b/kotlinx-coroutines-core/jvm/src/internal/ExceptionsConstructor.kt @@ -33,42 +33,48 @@ internal fun <E : Throwable> tryCopyException(exception: E): E? { private fun <E : Throwable> createConstructor(clz: Class<E>): Ctor { val nullResult: Ctor = { null } // Pre-cache class - // Skip reflective copy if an exception has additional fields (that are usually populated in user-defined constructors) + // Skip reflective copy if an exception has additional fields (that are typically populated in user-defined constructors) if (throwableFields != clz.fieldsCountOrDefault(0)) return nullResult /* - * Try to reflectively find constructor(), constructor(message, cause), constructor(cause) or constructor(message). - * Exceptions are shared among coroutines, so we should copy exception before recovering current stacktrace. - */ - val constructors = clz.constructors.sortedByDescending { it.parameterTypes.size } - for (constructor in constructors) { - val result = createSafeConstructor(constructor) - if (result != null) return result - } - return nullResult -} - -private fun createSafeConstructor(constructor: Constructor<*>): Ctor? { - val p = constructor.parameterTypes - return when (p.size) { - 2 -> when { - p[0] == String::class.java && p[1] == Throwable::class.java -> - safeCtor { e -> constructor.newInstance(e.message, e) as Throwable } - else -> null + * Try to reflectively find constructor(message, cause), constructor(message), constructor(cause), or constructor(), + * in that order of priority. + * Exceptions are shared among coroutines, so we should copy exception before recovering current stacktrace. + * + * By default, Java's reflection iterates over ctors in the source-code order and the sorting is stable, so we can + * not rely on the order of iteration. Instead, we assign a unique priority to each ctor type. + */ + return clz.constructors.map { constructor -> + val p = constructor.parameterTypes + when (p.size) { + 2 -> when { + p[0] == String::class.java && p[1] == Throwable::class.java -> + safeCtor { e -> constructor.newInstance(e.message, e) as Throwable } to 3 + else -> null to -1 + } + 1 -> when (p[0]) { + String::class.java -> + safeCtor { e -> (constructor.newInstance(e.message) as Throwable).also { it.initCause(e) } } to 2 + Throwable::class.java -> + safeCtor { e -> constructor.newInstance(e) as Throwable } to 1 + else -> null to -1 + } + 0 -> safeCtor { e -> (constructor.newInstance() as Throwable).also { it.initCause(e) } } to 0 + else -> null to -1 } - 1 -> when (p[0]) { - Throwable::class.java -> - safeCtor { e -> constructor.newInstance(e) as Throwable } - String::class.java -> - safeCtor { e -> (constructor.newInstance(e.message) as Throwable).also { it.initCause(e) } } - else -> null - } - 0 -> safeCtor { e -> (constructor.newInstance() as Throwable).also { it.initCause(e) } } - else -> null - } + }.maxByOrNull(Pair<*, Int>::second)?.first ?: nullResult } -private inline fun safeCtor(crossinline block: (Throwable) -> Throwable): Ctor = - { e -> runCatching { block(e) }.getOrNull() } +private fun safeCtor(block: (Throwable) -> Throwable): Ctor = { e -> + runCatching { + val result = block(e) + /* + * Verify that the new exception has the same message as the original one (bail out if not, see #1631) + * or if the new message complies the contract from `Throwable(cause).message` contract. + */ + if (e.message != result.message && result.message != e.toString()) null + else result + }.getOrNull() +} private fun Class<*>.fieldsCountOrDefault(defaultValue: Int) = kotlin.runCatching { fieldsCount() }.getOrDefault(defaultValue) diff --git a/kotlinx-coroutines-core/jvm/src/internal/MainDispatchers.kt b/kotlinx-coroutines-core/jvm/src/internal/MainDispatchers.kt index e87952b4..0c55d92e 100644 --- a/kotlinx-coroutines-core/jvm/src/internal/MainDispatchers.kt +++ b/kotlinx-coroutines-core/jvm/src/internal/MainDispatchers.kt @@ -98,9 +98,6 @@ private class MissingMainCoroutineDispatcher( override fun limitedParallelism(parallelism: Int): CoroutineDispatcher = missing() - override suspend fun delay(time: Long) = - missing() - override fun invokeOnTimeout(timeMillis: Long, block: Runnable, context: CoroutineContext): DisposableHandle = missing() diff --git a/kotlinx-coroutines-core/jvm/src/internal/ResizableAtomicArray.kt b/kotlinx-coroutines-core/jvm/src/internal/ResizableAtomicArray.kt index f949d9f5..6c98a9da 100644 --- a/kotlinx-coroutines-core/jvm/src/internal/ResizableAtomicArray.kt +++ b/kotlinx-coroutines-core/jvm/src/internal/ResizableAtomicArray.kt @@ -28,11 +28,12 @@ internal class ResizableAtomicArray<T>(initialLength: Int) { val curLen = curArray.length() if (index < curLen) { curArray[index] = value - } else { - val newArray = AtomicReferenceArray<T>((index + 1).coerceAtLeast(2 * curLen)) - for (i in 0 until curLen) newArray[i] = curArray[i] - newArray[index] = value - array = newArray // copy done + return } + // It would be nice to copy array in batch instead of 1 by 1, but it seems like Java has no API for that + val newArray = AtomicReferenceArray<T>((index + 1).coerceAtLeast(2 * curLen)) + for (i in 0 until curLen) newArray[i] = curArray[i] + newArray[index] = value + array = newArray // copy done } } diff --git a/kotlinx-coroutines-core/jvm/src/internal/StackTraceRecovery.kt b/kotlinx-coroutines-core/jvm/src/internal/StackTraceRecovery.kt index 6153862e..afc99896 100644 --- a/kotlinx-coroutines-core/jvm/src/internal/StackTraceRecovery.kt +++ b/kotlinx-coroutines-core/jvm/src/internal/StackTraceRecovery.kt @@ -7,6 +7,8 @@ package kotlinx.coroutines.internal import kotlinx.coroutines.* +import _COROUTINE.ARTIFICIAL_FRAME_PACKAGE_NAME +import _COROUTINE.ArtificialStackFrames import java.util.* import kotlin.coroutines.* import kotlin.coroutines.intrinsics.* @@ -18,6 +20,8 @@ import kotlin.coroutines.intrinsics.* private const val baseContinuationImplClass = "kotlin.coroutines.jvm.internal.BaseContinuationImpl" private const val stackTraceRecoveryClass = "kotlinx.coroutines.internal.StackTraceRecoveryKt" +private val ARTIFICIAL_FRAME = ArtificialStackFrames().coroutineBoundary() + private val baseContinuationImplClassName = runCatching { Class.forName(baseContinuationImplClass).canonicalName }.getOrElse { baseContinuationImplClass } @@ -29,20 +33,20 @@ private val stackTraceRecoveryClassName = runCatching { internal actual fun <E : Throwable> recoverStackTrace(exception: E): E { if (!RECOVER_STACK_TRACES) return exception // No unwrapping on continuation-less path: exception is not reported multiple times via slow paths - val copy = tryCopyAndVerify(exception) ?: return exception + val copy = tryCopyException(exception) ?: return exception return copy.sanitizeStackTrace() } private fun <E : Throwable> E.sanitizeStackTrace(): E { val stackTrace = stackTrace val size = stackTrace.size - val lastIntrinsic = stackTrace.frameIndex(stackTraceRecoveryClassName) + val lastIntrinsic = stackTrace.indexOfLast { stackTraceRecoveryClassName == it.className } val startIndex = lastIntrinsic + 1 - val endIndex = stackTrace.frameIndex(baseContinuationImplClassName) + val endIndex = stackTrace.firstFrameIndex(baseContinuationImplClassName) val adjustment = if (endIndex == -1) 0 else size - endIndex val trace = Array(size - lastIntrinsic - adjustment) { if (it == 0) { - artificialFrame("Coroutine boundary") + ARTIFICIAL_FRAME } else { stackTrace[startIndex + it - 1] } @@ -66,7 +70,7 @@ private fun <E : Throwable> recoverFromStackFrame(exception: E, continuation: Co val (cause, recoveredStacktrace) = exception.causeAndStacktrace() // Try to create an exception of the same type and get stacktrace from continuation - val newException = tryCopyAndVerify(cause) ?: return exception + val newException = tryCopyException(cause) ?: return exception // Update stacktrace val stacktrace = createStackTrace(continuation) if (stacktrace.isEmpty()) return exception @@ -78,14 +82,6 @@ private fun <E : Throwable> recoverFromStackFrame(exception: E, continuation: Co return createFinalException(cause, newException, stacktrace) } -private fun <E : Throwable> tryCopyAndVerify(exception: E): E? { - val newException = tryCopyException(exception) ?: return null - // Verify that the new exception has the same message as the original one (bail out if not, see #1631) - // CopyableThrowable has control over its message and thus can modify it the way it wants - if (exception !is CopyableThrowable<*> && newException.message != exception.message) return null - return newException -} - /* * Here we partially copy original exception stackTrace to make current one much prettier. * E.g. for @@ -97,15 +93,15 @@ private fun <E : Throwable> tryCopyAndVerify(exception: E): E? { * IllegalStateException * at foo * at kotlin.coroutines.resumeWith - * (Coroutine boundary) + * at _COROUTINE._BOUNDARY._(CoroutineDebugging.kt) * at bar * ...real stackTrace... * caused by "IllegalStateException" (original one) */ private fun <E : Throwable> createFinalException(cause: E, result: E, resultStackTrace: ArrayDeque<StackTraceElement>): E { - resultStackTrace.addFirst(artificialFrame("Coroutine boundary")) + resultStackTrace.addFirst(ARTIFICIAL_FRAME) val causeTrace = cause.stackTrace - val size = causeTrace.frameIndex(baseContinuationImplClassName) + val size = causeTrace.firstFrameIndex(baseContinuationImplClassName) if (size == -1) { result.stackTrace = resultStackTrace.toTypedArray() return result @@ -153,7 +149,6 @@ private fun mergeRecoveredTraces(recoveredStacktrace: Array<StackTraceElement>, } } -@Suppress("NOTHING_TO_INLINE") internal actual suspend inline fun recoverAndThrow(exception: Throwable): Nothing { if (!RECOVER_STACK_TRACES) throw exception suspendCoroutineUninterceptedOrReturn<Nothing> { @@ -193,13 +188,8 @@ private fun createStackTrace(continuation: CoroutineStackFrame): ArrayDeque<Stac return stack } -/** - * @suppress - */ -@InternalCoroutinesApi -public fun artificialFrame(message: String): StackTraceElement = java.lang.StackTraceElement("\b\b\b($message", "\b", "\b", -1) -internal fun StackTraceElement.isArtificial() = className.startsWith("\b\b\b") -private fun Array<StackTraceElement>.frameIndex(methodName: String) = indexOfFirst { methodName == it.className } +internal fun StackTraceElement.isArtificial() = className.startsWith(ARTIFICIAL_FRAME_PACKAGE_NAME) +private fun Array<StackTraceElement>.firstFrameIndex(methodName: String) = indexOfFirst { methodName == it.className } private fun StackTraceElement.elementWiseEquals(e: StackTraceElement): Boolean { /* diff --git a/kotlinx-coroutines-core/jvm/src/internal/Synchronized.kt b/kotlinx-coroutines-core/jvm/src/internal/Synchronized.kt index 6284f3a0..5b12faaf 100644 --- a/kotlinx-coroutines-core/jvm/src/internal/Synchronized.kt +++ b/kotlinx-coroutines-core/jvm/src/internal/Synchronized.kt @@ -16,5 +16,5 @@ public actual typealias SynchronizedObject = Any * @suppress **This an internal API and should not be used from general code.** */ @InternalCoroutinesApi -public actual inline fun <T> synchronized(lock: SynchronizedObject, block: () -> T): T = +public actual inline fun <T> synchronizedImpl(lock: SynchronizedObject, block: () -> T): T = kotlin.synchronized(lock, block) diff --git a/kotlinx-coroutines-core/jvm/src/internal/ThreadLocal.kt b/kotlinx-coroutines-core/jvm/src/internal/ThreadLocal.kt index 0207334a..0924a5b3 100644 --- a/kotlinx-coroutines-core/jvm/src/internal/ThreadLocal.kt +++ b/kotlinx-coroutines-core/jvm/src/internal/ThreadLocal.kt @@ -8,3 +8,5 @@ import java.lang.ThreadLocal @Suppress("ACTUAL_WITHOUT_EXPECT") // internal visibility internal actual typealias CommonThreadLocal<T> = ThreadLocal<T> + +internal actual fun<T> commonThreadLocal(name: Symbol): CommonThreadLocal<T> = ThreadLocal() diff --git a/kotlinx-coroutines-core/jvm/src/module-info.java b/kotlinx-coroutines-core/jvm/src/module-info.java new file mode 100644 index 00000000..2759a342 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/src/module-info.java @@ -0,0 +1,29 @@ +import kotlinx.coroutines.CoroutineExceptionHandler; +import kotlinx.coroutines.internal.MainDispatcherFactory; + +module kotlinx.coroutines.core { + requires transitive kotlin.stdlib; + requires kotlinx.atomicfu; + + // these are used by kotlinx.coroutines.debug.AgentPremain + requires static java.instrument; // contains java.lang.instrument.* + requires static jdk.unsupported; // contains sun.misc.Signal + + exports kotlinx.coroutines; + exports kotlinx.coroutines.channels; + exports kotlinx.coroutines.debug; + exports kotlinx.coroutines.debug.internal; + exports kotlinx.coroutines.flow; + exports kotlinx.coroutines.flow.internal; + exports kotlinx.coroutines.future; + exports kotlinx.coroutines.internal; + exports kotlinx.coroutines.intrinsics; + exports kotlinx.coroutines.scheduling; + exports kotlinx.coroutines.selects; + exports kotlinx.coroutines.stream; + exports kotlinx.coroutines.sync; + exports kotlinx.coroutines.time; + + uses CoroutineExceptionHandler; + uses MainDispatcherFactory; +} diff --git a/kotlinx-coroutines-core/jvm/src/scheduling/CoroutineScheduler.kt b/kotlinx-coroutines-core/jvm/src/scheduling/CoroutineScheduler.kt index e08d1cef..5ca3de57 100644 --- a/kotlinx-coroutines-core/jvm/src/scheduling/CoroutineScheduler.kt +++ b/kotlinx-coroutines-core/jvm/src/scheduling/CoroutineScheduler.kt @@ -10,15 +10,16 @@ import kotlinx.coroutines.internal.* import java.io.* import java.util.concurrent.* import java.util.concurrent.locks.* +import kotlin.jvm.internal.Ref.ObjectRef import kotlin.math.* import kotlin.random.* /** * Coroutine scheduler (pool of shared threads) which primary target is to distribute dispatched coroutines - * over worker threads, including both CPU-intensive and blocking tasks, is the most efficient manner. + * over worker threads, including both CPU-intensive and blocking tasks, in the most efficient manner. * * Current scheduler implementation has two optimization targets: - * * Efficiency in the face of communication patterns (e.g., actors communicating via channel) + * * Efficiency in the face of communication patterns (e.g. actors communicating via channel) * * Dynamic resizing to support blocking calls without re-dispatching coroutine to separate "blocking" thread pool. * * ### Structural overview @@ -69,7 +70,7 @@ import kotlin.random.* * * When a new task arrives in the scheduler (whether it is local or global queue), * either an idle worker is being signalled, or a new worker is attempted to be created. - * Only [corePoolSize] workers can be created for regular CPU tasks) + * (Only [corePoolSize] workers can be created for regular CPU tasks) * * ### Support for blocking tasks * The scheduler also supports the notion of [blocking][TASK_PROBABLY_BLOCKING] tasks. @@ -251,28 +252,37 @@ internal class CoroutineScheduler( /** * State of worker threads. - * [workers] is array of lazily created workers up to [maxPoolSize] workers. + * [workers] is a dynamic array of lazily created workers up to [maxPoolSize] workers. * [createdWorkers] is count of already created workers (worker with index lesser than [createdWorkers] exists). - * [blockingTasks] is count of pending (either in the queue or being executed) tasks + * [blockingTasks] is count of pending (either in the queue or being executed) blocking tasks. + * + * Workers array is also used as a lock for workers' creation and termination sequence. * * **NOTE**: `workers[0]` is always `null` (never used, works as sentinel value), so * workers are 1-indexed, code path in [Worker.trySteal] is a bit faster and index swap during termination - * works properly + * works properly. + * + * Initial size is `Dispatchers.Default` size * 2 to prevent unnecessary resizes for slightly or steadily loaded + * applications. */ @JvmField - val workers = ResizableAtomicArray<Worker>(corePoolSize + 1) + val workers = ResizableAtomicArray<Worker>((corePoolSize + 1) * 2) /** - * Long describing state of workers in this pool. - * Currently includes created, CPU-acquired and blocking workers each occupying [BLOCKING_SHIFT] bits. + * The `Long` value describing the state of workers in this pool. + * Currently, includes created, CPU-acquired, and blocking workers, each occupying [BLOCKING_SHIFT] bits. + * + * State layout (highest to lowest bits): + * | --- number of cpu permits, 22 bits --- | --- number of blocking tasks, 21 bits --- | --- number of created threads, 21 bits --- | */ private val controlState = atomic(corePoolSize.toLong() shl CPU_PERMITS_SHIFT) + private val createdWorkers: Int inline get() = (controlState.value and CREATED_MASK).toInt() private val availableCpuPermits: Int inline get() = availableCpuPermits(controlState.value) private inline fun createdWorkers(state: Long): Int = (state and CREATED_MASK).toInt() private inline fun blockingTasks(state: Long): Int = (state and BLOCKING_MASK shr BLOCKING_SHIFT).toInt() - public inline fun availableCpuPermits(state: Long): Int = (state and CPU_PERMITS_MASK shr CPU_PERMITS_SHIFT).toInt() + inline fun availableCpuPermits(state: Long): Int = (state and CPU_PERMITS_MASK shr CPU_PERMITS_SHIFT).toInt() // Guarded by synchronization private inline fun incrementCreatedWorkers(): Int = createdWorkers(controlState.incrementAndGet()) @@ -382,6 +392,10 @@ internal class CoroutineScheduler( fun dispatch(block: Runnable, taskContext: TaskContext = NonBlockingContext, tailDispatch: Boolean = false) { trackTask() // this is needed for virtual time support val task = createTask(block, taskContext) + val isBlockingTask = task.isBlocking + // Invariant: we increment counter **before** publishing the task + // so executing thread can safely decrement the number of blocking tasks + val stateSnapshot = if (isBlockingTask) incrementBlockingTasks() else 0 // try to submit the task to the local queue and act depending on the result val currentWorker = currentWorker() val notAdded = currentWorker.submitToLocalQueue(task, tailDispatch) @@ -393,12 +407,12 @@ internal class CoroutineScheduler( } val skipUnpark = tailDispatch && currentWorker != null // Checking 'task' instead of 'notAdded' is completely okay - if (task.mode == TASK_NON_BLOCKING) { + if (isBlockingTask) { + // Use state snapshot to better estimate the number of running threads + signalBlockingWork(stateSnapshot, skipUnpark = skipUnpark) + } else { if (skipUnpark) return signalCpuWork() - } else { - // Increment blocking tasks anyway - signalBlockingWork(skipUnpark = skipUnpark) } } @@ -412,11 +426,11 @@ internal class CoroutineScheduler( return TaskImpl(block, nanoTime, taskContext) } - private fun signalBlockingWork(skipUnpark: Boolean) { - // Use state snapshot to avoid thread overprovision - val stateSnapshot = incrementBlockingTasks() + // NB: should only be called from 'dispatch' method due to blocking tasks increment + private fun signalBlockingWork(stateSnapshot: Long, skipUnpark: Boolean) { if (skipUnpark) return if (tryUnpark()) return + // Use state snapshot to avoid accidental thread overprovision if (tryCreateWorker(stateSnapshot)) return tryUnpark() // Try unpark again in case there was race between permit release and parking } @@ -455,12 +469,13 @@ internal class CoroutineScheduler( } } - /* + /** * Returns the number of CPU workers after this function (including new worker) or * 0 if no worker was created. */ private fun createNewWorker(): Int { - synchronized(workers) { + val worker: Worker + return synchronized(workers) { // Make sure we're not trying to resurrect terminated scheduler if (isTerminated) return -1 val state = controlState.value @@ -478,12 +493,11 @@ internal class CoroutineScheduler( * 2) Make it observable by increment created workers count * 3) Only then start the worker, otherwise it may miss its own creation */ - val worker = Worker(newIndex) + worker = Worker(newIndex) workers.setSynchronized(newIndex, worker) require(newIndex == incrementCreatedWorkers()) - worker.start() - return cpuWorkers + 1 - } + cpuWorkers + 1 + }.also { worker.start() } // Start worker when the lock is released to reduce contention, see #3652 } /** @@ -599,6 +613,12 @@ internal class CoroutineScheduler( val localQueue: WorkQueue = WorkQueue() /** + * Slot that is used to steal tasks into to avoid re-adding them + * to the local queue. See [trySteal] + */ + private val stolenTask: ObjectRef<Task?> = ObjectRef() + + /** * Worker state. **Updated only by this worker thread**. * By default, worker is in DORMANT state in the case when it was created, but all CPU tokens or tasks were taken. * Is used locally by the worker to maintain its own invariants. @@ -617,7 +637,7 @@ internal class CoroutineScheduler( /** * It is set to the termination deadline when started doing [park] and it reset - * when there is a task. It servers as protection against spurious wakeups of parkNanos. + * when there is a task. It serves as protection against spurious wakeups of parkNanos. */ private var terminationDeadline = 0L @@ -713,13 +733,36 @@ internal class CoroutineScheduler( tryReleaseCpu(WorkerState.TERMINATED) } + /** + * See [runSingleTaskFromCurrentSystemDispatcher] for rationale and details. + * This is a fine-tailored method for a specific use-case not expected to be used widely. + */ + fun runSingleTask(): Long { + val stateSnapshot = state + val isCpuThread = state == WorkerState.CPU_ACQUIRED + val task = if (isCpuThread) { + findCpuTask() + } else { + findBlockingTask() + } + if (task == null) { + if (minDelayUntilStealableTaskNs == 0L) return -1L + return minDelayUntilStealableTaskNs + } + runSafely(task) + if (!isCpuThread) decrementBlockingTasks() + assert { state == stateSnapshot} + return 0L + } + + fun isIo() = state == WorkerState.BLOCKING + // Counterpart to "tryUnpark" private fun tryPark() { if (!inStack()) { parkedWorkersStackPush(this) return } - assert { localQueue.size == 0 } workerCtl.value = PARKED // Update value once /* * inStack() prevents spurious wakeups, while workerCtl.value == PARKED @@ -866,15 +909,28 @@ internal class CoroutineScheduler( } } - fun findTask(scanLocalQueue: Boolean): Task? { - if (tryAcquireCpuPermit()) return findAnyTask(scanLocalQueue) - // If we can't acquire a CPU permit -- attempt to find blocking task - val task = if (scanLocalQueue) { - localQueue.poll() ?: globalBlockingQueue.removeFirstOrNull() - } else { - globalBlockingQueue.removeFirstOrNull() - } - return task ?: trySteal(blockingOnly = true) + fun findTask(mayHaveLocalTasks: Boolean): Task? { + if (tryAcquireCpuPermit()) return findAnyTask(mayHaveLocalTasks) + /* + * If we can't acquire a CPU permit, attempt to find blocking task: + * * Check if our queue has one (maybe mixed in with CPU tasks) + * * Poll global and try steal + */ + return findBlockingTask() + } + + // NB: ONLY for runSingleTask method + private fun findBlockingTask(): Task? { + return localQueue.pollBlocking() + ?: globalBlockingQueue.removeFirstOrNull() + ?: trySteal(STEAL_BLOCKING_ONLY) + } + + // NB: ONLY for runSingleTask method + private fun findCpuTask(): Task? { + return localQueue.pollCpu() + ?: globalBlockingQueue.removeFirstOrNull() + ?: trySteal(STEAL_CPU_ONLY) } private fun findAnyTask(scanLocalQueue: Boolean): Task? { @@ -890,7 +946,7 @@ internal class CoroutineScheduler( } else { pollGlobalQueues()?.let { return it } } - return trySteal(blockingOnly = false) + return trySteal(STEAL_ANY) } private fun pollGlobalQueues(): Task? { @@ -903,8 +959,7 @@ internal class CoroutineScheduler( } } - private fun trySteal(blockingOnly: Boolean): Task? { - assert { localQueue.size == 0 } + private fun trySteal(stealingMode: StealingMode): Task? { val created = createdWorkers // 0 to await an initialization and 1 to avoid excess stealing on single-core machines if (created < 2) { @@ -918,14 +973,11 @@ internal class CoroutineScheduler( if (currentIndex > created) currentIndex = 1 val worker = workers[currentIndex] if (worker !== null && worker !== this) { - assert { localQueue.size == 0 } - val stealResult = if (blockingOnly) { - localQueue.tryStealBlockingFrom(victim = worker.localQueue) - } else { - localQueue.tryStealFrom(victim = worker.localQueue) - } + val stealResult = worker.localQueue.trySteal(stealingMode, stolenTask) if (stealResult == TASK_STOLEN) { - return localQueue.poll() + val result = stolenTask.element + stolenTask.element = null + return result } else if (stealResult > 0) { minDelay = min(minDelay, stealResult) } diff --git a/kotlinx-coroutines-core/jvm/src/scheduling/Dispatcher.kt b/kotlinx-coroutines-core/jvm/src/scheduling/Dispatcher.kt index d55edec9..f91125a2 100644 --- a/kotlinx-coroutines-core/jvm/src/scheduling/Dispatcher.kt +++ b/kotlinx-coroutines-core/jvm/src/scheduling/Dispatcher.kt @@ -14,6 +14,14 @@ internal object DefaultScheduler : SchedulerCoroutineDispatcher( CORE_POOL_SIZE, MAX_POOL_SIZE, IDLE_WORKER_KEEP_ALIVE_NS, DEFAULT_SCHEDULER_NAME ) { + + @ExperimentalCoroutinesApi + override fun limitedParallelism(parallelism: Int): CoroutineDispatcher { + parallelism.checkParallelism() + if (parallelism >= CORE_POOL_SIZE) return this + return super.limitedParallelism(parallelism) + } + // Shuts down the dispatcher, used only by Dispatchers.shutdown() internal fun shutdown() { super.close() @@ -38,6 +46,13 @@ private object UnlimitedIoScheduler : CoroutineDispatcher() { override fun dispatch(context: CoroutineContext, block: Runnable) { DefaultScheduler.dispatchWithContext(block, BlockingContext, false) } + + @ExperimentalCoroutinesApi + override fun limitedParallelism(parallelism: Int): CoroutineDispatcher { + parallelism.checkParallelism() + if (parallelism >= MAX_POOL_SIZE) return this + return super.limitedParallelism(parallelism) + } } // Dispatchers.IO diff --git a/kotlinx-coroutines-core/jvm/src/scheduling/Tasks.kt b/kotlinx-coroutines-core/jvm/src/scheduling/Tasks.kt index 5403cfc1..5249edf7 100644 --- a/kotlinx-coroutines-core/jvm/src/scheduling/Tasks.kt +++ b/kotlinx-coroutines-core/jvm/src/scheduling/Tasks.kt @@ -9,8 +9,13 @@ import kotlinx.coroutines.internal.* import java.util.concurrent.* -// Internal debuggability name + thread name prefixes -internal const val DEFAULT_SCHEDULER_NAME = "DefaultDispatcher" +/** + * The name of the default scheduler. The names of the worker threads of [Dispatchers.Default] have it as their prefix. + */ +@JvmField +internal val DEFAULT_SCHEDULER_NAME = systemProp( + "kotlinx.coroutines.scheduler.default.name", "DefaultDispatcher" +) // 100us as default @JvmField @@ -74,12 +79,15 @@ internal val NonBlockingContext: TaskContext = TaskContextImpl(TASK_NON_BLOCKING @JvmField internal val BlockingContext: TaskContext = TaskContextImpl(TASK_PROBABLY_BLOCKING) -internal abstract class Task( +@PublishedApi +internal abstract class Task internal constructor( + // Used by the IDEA debugger via reflection and must be kept binary-compatible, see KTIJ-24102 @JvmField var submissionTime: Long, - @JvmField var taskContext: TaskContext + // Used by the IDEA debugger via reflection and must be kept binary-compatible, see KTIJ-24102 + @JvmField internal var taskContext: TaskContext ) : Runnable { - constructor() : this(0, NonBlockingContext) - inline val mode: Int get() = taskContext.taskMode // TASK_XXX + internal constructor() : this(0, NonBlockingContext) + internal inline val mode: Int get() = taskContext.taskMode // TASK_XXX } internal inline val Task.isBlocking get() = taskContext.taskMode == TASK_PROBABLY_BLOCKING diff --git a/kotlinx-coroutines-core/jvm/src/scheduling/WorkQueue.kt b/kotlinx-coroutines-core/jvm/src/scheduling/WorkQueue.kt index 6a9a8a5a..a185410a 100644 --- a/kotlinx-coroutines-core/jvm/src/scheduling/WorkQueue.kt +++ b/kotlinx-coroutines-core/jvm/src/scheduling/WorkQueue.kt @@ -7,6 +7,7 @@ package kotlinx.coroutines.scheduling import kotlinx.atomicfu.* import kotlinx.coroutines.* import java.util.concurrent.atomic.* +import kotlin.jvm.internal.Ref.ObjectRef internal const val BUFFER_CAPACITY_BASE = 7 internal const val BUFFER_CAPACITY = 1 shl BUFFER_CAPACITY_BASE @@ -15,6 +16,14 @@ internal const val MASK = BUFFER_CAPACITY - 1 // 128 by default internal const val TASK_STOLEN = -1L internal const val NOTHING_TO_STEAL = -2L +internal typealias StealingMode = Int +internal const val STEAL_ANY: StealingMode = 3 +internal const val STEAL_CPU_ONLY: StealingMode = 2 +internal const val STEAL_BLOCKING_ONLY: StealingMode = 1 + +internal inline val Task.maskForStealingMode: Int + get() = if (isBlocking) STEAL_BLOCKING_ONLY else STEAL_CPU_ONLY + /** * Tightly coupled with [CoroutineScheduler] queue of pending tasks, but extracted to separate file for simplicity. * At any moment queue is used only by [CoroutineScheduler.Worker] threads, has only one producer (worker owning this queue) @@ -31,7 +40,7 @@ internal const val NOTHING_TO_STEAL = -2L * (scheduler workers without a CPU permit steal blocking tasks via this mechanism). Such property enforces us to use CAS in * order to properly claim value from the buffer. * Moreover, [Task] objects are reusable, so it may seem that this queue is prone to ABA problem. - * Indeed it formally has ABA-problem, but the whole processing logic is written in the way that such ABA is harmless. + * Indeed, it formally has ABA-problem, but the whole processing logic is written in the way that such ABA is harmless. * I have discovered a truly marvelous proof of this, which this KDoc is too narrow to contain. */ internal class WorkQueue { @@ -46,10 +55,12 @@ internal class WorkQueue { * [T2] changeProducerIndex (3) * [T3] changeConsumerIndex (4) * - * Which can lead to resulting size bigger than actual size at any moment of time. - * This is in general harmless because steal will be blocked by timer + * Which can lead to resulting size being negative or bigger than actual size at any moment of time. + * This is in general harmless because steal will be blocked by timer. + * Negative sizes can be observed only when non-owner reads the size, which happens only + * for diagnostic toString(). */ - internal val bufferSize: Int get() = producerIndex.value - consumerIndex.value + private val bufferSize: Int get() = producerIndex.value - consumerIndex.value internal val size: Int get() = if (lastScheduledTask.value != null) bufferSize + 1 else bufferSize private val buffer: AtomicReferenceArray<Task?> = AtomicReferenceArray(BUFFER_CAPACITY) private val lastScheduledTask = atomic<Task?>(null) @@ -80,8 +91,8 @@ internal class WorkQueue { * `null` if task was added, task that wasn't added otherwise. */ private fun addLast(task: Task): Task? { - if (task.isBlocking) blockingTasksInBuffer.incrementAndGet() if (bufferSize == BUFFER_CAPACITY - 1) return task + if (task.isBlocking) blockingTasksInBuffer.incrementAndGet() val nextIndex = producerIndex.value and MASK /* * If current element is not null then we're racing with a really slow consumer that committed the consumer index, @@ -100,41 +111,82 @@ internal class WorkQueue { } /** - * Tries stealing from [victim] queue into this queue. + * Tries stealing from this queue into the [stolenTaskRef] argument. * * Returns [NOTHING_TO_STEAL] if queue has nothing to steal, [TASK_STOLEN] if at least task was stolen * or positive value of how many nanoseconds should pass until the head of this queue will be available to steal. + * + * [StealingMode] controls what tasks to steal: + * * [STEAL_ANY] is default mode for scheduler, task from the head (in FIFO order) is stolen + * * [STEAL_BLOCKING_ONLY] is mode for stealing *an arbitrary* blocking task, which is used by the scheduler when helping in Dispatchers.IO mode + * * [STEAL_CPU_ONLY] is a kludge for `runSingleTaskFromCurrentSystemDispatcher` */ - fun tryStealFrom(victim: WorkQueue): Long { - assert { bufferSize == 0 } - val task = victim.pollBuffer() + fun trySteal(stealingMode: StealingMode, stolenTaskRef: ObjectRef<Task?>): Long { + val task = when (stealingMode) { + STEAL_ANY -> pollBuffer() + else -> stealWithExclusiveMode(stealingMode) + } + if (task != null) { - val notAdded = add(task) - assert { notAdded == null } + stolenTaskRef.element = task return TASK_STOLEN } - return tryStealLastScheduled(victim, blockingOnly = false) + return tryStealLastScheduled(stealingMode, stolenTaskRef) } - fun tryStealBlockingFrom(victim: WorkQueue): Long { - assert { bufferSize == 0 } - var start = victim.consumerIndex.value - val end = victim.producerIndex.value - val buffer = victim.buffer + // Steal only tasks of a particular kind, potentially invoking full queue scan + private fun stealWithExclusiveMode(stealingMode: StealingMode): Task? { + var start = consumerIndex.value + val end = producerIndex.value + val onlyBlocking = stealingMode == STEAL_BLOCKING_ONLY + // Bail out if there is no blocking work for us + while (start != end) { + if (onlyBlocking && blockingTasksInBuffer.value == 0) return null + return tryExtractFromTheMiddle(start++, onlyBlocking) ?: continue + } + return null + } + + // Polls for blocking task, invoked only by the owner + // NB: ONLY for runSingleTask method + fun pollBlocking(): Task? = pollWithExclusiveMode(onlyBlocking = true /* only blocking */) + + // Polls for CPU task, invoked only by the owner + // NB: ONLY for runSingleTask method + fun pollCpu(): Task? = pollWithExclusiveMode(onlyBlocking = false /* only cpu */) + + private fun pollWithExclusiveMode(/* Only blocking OR only CPU */ onlyBlocking: Boolean): Task? { + while (true) { // Poll the slot + val lastScheduled = lastScheduledTask.value ?: break + if (lastScheduled.isBlocking != onlyBlocking) break + if (lastScheduledTask.compareAndSet(lastScheduled, null)) { + return lastScheduled + } // Failed -> someone else stole it + } + + // Failed to poll the slot, scan the queue + val start = consumerIndex.value + var end = producerIndex.value + // Bail out if there is no blocking work for us while (start != end) { - val index = start and MASK - if (victim.blockingTasksInBuffer.value == 0) break - val value = buffer[index] - if (value != null && value.isBlocking && buffer.compareAndSet(index, value, null)) { - victim.blockingTasksInBuffer.decrementAndGet() - add(value) - return TASK_STOLEN - } else { - ++start + if (onlyBlocking && blockingTasksInBuffer.value == 0) return null + val task = tryExtractFromTheMiddle(--end, onlyBlocking) + if (task != null) { + return task } } - return tryStealLastScheduled(victim, blockingOnly = true) + return null + } + + private fun tryExtractFromTheMiddle(index: Int, onlyBlocking: Boolean): Task? { + val arrayIndex = index and MASK + val value = buffer[arrayIndex] + if (value != null && value.isBlocking == onlyBlocking && buffer.compareAndSet(arrayIndex, value, null)) { + if (onlyBlocking) blockingTasksInBuffer.decrementAndGet() + return value + } + return null } fun offloadAllWorkTo(globalQueue: GlobalQueue) { @@ -145,12 +197,14 @@ internal class WorkQueue { } /** - * Contract on return value is the same as for [tryStealFrom] + * Contract on return value is the same as for [trySteal] */ - private fun tryStealLastScheduled(victim: WorkQueue, blockingOnly: Boolean): Long { + private fun tryStealLastScheduled(stealingMode: StealingMode, stolenTaskRef: ObjectRef<Task?>): Long { while (true) { - val lastScheduled = victim.lastScheduledTask.value ?: return NOTHING_TO_STEAL - if (blockingOnly && !lastScheduled.isBlocking) return NOTHING_TO_STEAL + val lastScheduled = lastScheduledTask.value ?: return NOTHING_TO_STEAL + if ((lastScheduled.maskForStealingMode and stealingMode) == 0) { + return NOTHING_TO_STEAL + } // TODO time wraparound ? val time = schedulerTimeSource.nanoTime() @@ -163,8 +217,8 @@ internal class WorkQueue { * If CAS has failed, either someone else had stolen this task or the owner executed this task * and dispatched another one. In the latter case we should retry to avoid missing task. */ - if (victim.lastScheduledTask.compareAndSet(lastScheduled, null)) { - add(lastScheduled) + if (lastScheduledTask.compareAndSet(lastScheduled, null)) { + stolenTaskRef.element = lastScheduled return TASK_STOLEN } continue diff --git a/kotlinx-coroutines-core/jvm/test-resources/stacktraces/channels/testOfferFromScope.txt b/kotlinx-coroutines-core/jvm/test-resources/stacktraces/channels/testOfferFromScope.txt index bf3fd3a3..aa5a6a17 100644 --- a/kotlinx-coroutines-core/jvm/test-resources/stacktraces/channels/testOfferFromScope.txt +++ b/kotlinx-coroutines-core/jvm/test-resources/stacktraces/channels/testOfferFromScope.txt @@ -1,10 +1,10 @@ kotlinx.coroutines.RecoverableTestException at kotlinx.coroutines.exceptions.StackTraceRecoveryChannelsTest$testOfferFromScope$1.invokeSuspend(StackTraceRecoveryChannelsTest.kt:109) - (Coroutine boundary) + at _COROUTINE._BOUNDARY._(CoroutineDebugging.kt) at kotlinx.coroutines.exceptions.StackTraceRecoveryChannelsTest.sendInChannel(StackTraceRecoveryChannelsTest.kt:167) at kotlinx.coroutines.exceptions.StackTraceRecoveryChannelsTest$sendWithContext$2.invokeSuspend(StackTraceRecoveryChannelsTest.kt:162) at kotlinx.coroutines.exceptions.StackTraceRecoveryChannelsTest$sendFromScope$2.invokeSuspend(StackTraceRecoveryChannelsTest.kt:172) at kotlinx.coroutines.exceptions.StackTraceRecoveryChannelsTest$testOfferFromScope$1.invokeSuspend(StackTraceRecoveryChannelsTest.kt:112) Caused by: kotlinx.coroutines.RecoverableTestException at kotlinx.coroutines.exceptions.StackTraceRecoveryChannelsTest$testOfferFromScope$1.invokeSuspend(StackTraceRecoveryChannelsTest.kt:109) - at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
\ No newline at end of file + at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33) diff --git a/kotlinx-coroutines-core/jvm/test-resources/stacktraces/channels/testOfferWithContextWrapped.txt b/kotlinx-coroutines-core/jvm/test-resources/stacktraces/channels/testOfferWithContextWrapped.txt index 612d00de..4908d3be 100644 --- a/kotlinx-coroutines-core/jvm/test-resources/stacktraces/channels/testOfferWithContextWrapped.txt +++ b/kotlinx-coroutines-core/jvm/test-resources/stacktraces/channels/testOfferWithContextWrapped.txt @@ -1,6 +1,6 @@ kotlinx.coroutines.RecoverableTestException at kotlinx.coroutines.exceptions.StackTraceRecoveryChannelsTest$testOfferWithContextWrapped$1.invokeSuspend(StackTraceRecoveryChannelsTest.kt:98) - (Coroutine boundary) + at _COROUTINE._BOUNDARY._(CoroutineDebugging.kt) at kotlinx.coroutines.exceptions.StackTraceRecoveryChannelsTest.sendInChannel(StackTraceRecoveryChannelsTest.kt:199) at kotlinx.coroutines.exceptions.StackTraceRecoveryChannelsTest$sendWithContext$2.invokeSuspend(StackTraceRecoveryChannelsTest.kt:194) at kotlinx.coroutines.exceptions.StackTraceRecoveryChannelsTest$testOfferWithContextWrapped$1.invokeSuspend(StackTraceRecoveryChannelsTest.kt:100) diff --git a/kotlinx-coroutines-core/jvm/test-resources/stacktraces/channels/testOfferWithCurrentContext.txt b/kotlinx-coroutines-core/jvm/test-resources/stacktraces/channels/testOfferWithCurrentContext.txt index 833afbf8..1eb464c7 100644 --- a/kotlinx-coroutines-core/jvm/test-resources/stacktraces/channels/testOfferWithCurrentContext.txt +++ b/kotlinx-coroutines-core/jvm/test-resources/stacktraces/channels/testOfferWithCurrentContext.txt @@ -1,6 +1,6 @@ kotlinx.coroutines.RecoverableTestException at kotlinx.coroutines.exceptions.StackTraceRecoveryChannelsTest$testOfferWithCurrentContext$1.invokeSuspend(StackTraceRecoveryChannelsTest.kt:86) - (Coroutine boundary) + at _COROUTINE._BOUNDARY._(CoroutineDebugging.kt) at kotlinx.coroutines.exceptions.StackTraceRecoveryChannelsTest.sendInChannel(StackTraceRecoveryChannelsTest.kt:210) at kotlinx.coroutines.exceptions.StackTraceRecoveryChannelsTest$sendWithContext$2.invokeSuspend(StackTraceRecoveryChannelsTest.kt:205) at kotlinx.coroutines.exceptions.StackTraceRecoveryChannelsTest$testOfferWithCurrentContext$1.invokeSuspend(StackTraceRecoveryChannelsTest.kt:89) diff --git a/kotlinx-coroutines-core/jvm/test-resources/stacktraces/channels/testReceiveFromChannel.txt b/kotlinx-coroutines-core/jvm/test-resources/stacktraces/channels/testReceiveFromChannel.txt index 66bb5e5e..da3558ba 100644 --- a/kotlinx-coroutines-core/jvm/test-resources/stacktraces/channels/testReceiveFromChannel.txt +++ b/kotlinx-coroutines-core/jvm/test-resources/stacktraces/channels/testReceiveFromChannel.txt @@ -1,8 +1,9 @@ kotlinx.coroutines.RecoverableTestException - at kotlinx.coroutines.exceptions.StackTraceRecoveryChannelsTest$testReceiveFromChannel$1$job$1.invokeSuspend(StackTraceRecoveryChannelsTest.kt:97) - (Coroutine boundary) - at kotlinx.coroutines.exceptions.StackTraceRecoveryChannelsTest.channelReceive(StackTraceRecoveryChannelsTest.kt:116) - at kotlinx.coroutines.exceptions.StackTraceRecoveryChannelsTest$testReceiveFromChannel$1.invokeSuspend(StackTraceRecoveryChannelsTest.kt:101) + at kotlinx.coroutines.channels.BufferedChannel.receive$suspendImpl(BufferedChannel.kt) + at kotlinx.coroutines.channels.BufferedChannel.receive(BufferedChannel.kt) + at kotlinx.coroutines.exceptions.StackTraceRecoveryChannelsTest.channelReceive(StackTraceRecoveryChannelsTest.kt) + at kotlinx.coroutines.exceptions.StackTraceRecoveryChannelsTest.access$channelReceive(StackTraceRecoveryChannelsTest.kt) + at kotlinx.coroutines.exceptions.StackTraceRecoveryChannelsTest$channelReceive$1.invokeSuspend(StackTraceRecoveryChannelsTest.kt) Caused by: kotlinx.coroutines.RecoverableTestException - at kotlinx.coroutines.exceptions.StackTraceRecoveryChannelsTest$testReceiveFromChannel$1$job$1.invokeSuspend(StackTraceRecoveryChannelsTest.kt:97) - at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
\ No newline at end of file + at kotlinx.coroutines.exceptions.StackTraceRecoveryChannelsTest$testReceiveFromChannel$1$job$1.invokeSuspend(StackTraceRecoveryChannelsTest.kt) + at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt) diff --git a/kotlinx-coroutines-core/jvm/test-resources/stacktraces/channels/testReceiveFromClosedChannel.txt b/kotlinx-coroutines-core/jvm/test-resources/stacktraces/channels/testReceiveFromClosedChannel.txt index 76c0b1a8..3f392cd3 100644 --- a/kotlinx-coroutines-core/jvm/test-resources/stacktraces/channels/testReceiveFromClosedChannel.txt +++ b/kotlinx-coroutines-core/jvm/test-resources/stacktraces/channels/testReceiveFromClosedChannel.txt @@ -1,8 +1,8 @@ kotlinx.coroutines.RecoverableTestException at kotlinx.coroutines.exceptions.StackTraceRecoveryChannelsTest$testReceiveFromClosedChannel$1.invokeSuspend(StackTraceRecoveryChannelsTest.kt:110) - (Coroutine boundary) + at _COROUTINE._BOUNDARY._(CoroutineDebugging.kt) at kotlinx.coroutines.exceptions.StackTraceRecoveryChannelsTest.channelReceive(StackTraceRecoveryChannelsTest.kt:116) at kotlinx.coroutines.exceptions.StackTraceRecoveryChannelsTest$testReceiveFromClosedChannel$1.invokeSuspend(StackTraceRecoveryChannelsTest.kt:111) Caused by: kotlinx.coroutines.RecoverableTestException at kotlinx.coroutines.exceptions.StackTraceRecoveryChannelsTest$testReceiveFromClosedChannel$1.invokeSuspend(StackTraceRecoveryChannelsTest.kt:110) - at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
\ No newline at end of file + at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33) diff --git a/kotlinx-coroutines-core/jvm/test-resources/stacktraces/channels/testSendFromScope.txt b/kotlinx-coroutines-core/jvm/test-resources/stacktraces/channels/testSendFromScope.txt index 9f932032..49c3628b 100644 --- a/kotlinx-coroutines-core/jvm/test-resources/stacktraces/channels/testSendFromScope.txt +++ b/kotlinx-coroutines-core/jvm/test-resources/stacktraces/channels/testSendFromScope.txt @@ -1,10 +1,10 @@ kotlinx.coroutines.RecoverableTestCancellationException at kotlinx.coroutines.exceptions.StackTraceRecoveryChannelsTest$testSendFromScope$1.invokeSuspend(StackTraceRecoveryChannelsTest.kt:136) - (Coroutine boundary) + at _COROUTINE._BOUNDARY._(CoroutineDebugging.kt) at kotlinx.coroutines.exceptions.StackTraceRecoveryChannelsTest.sendInChannel(StackTraceRecoveryChannelsTest.kt:167) at kotlinx.coroutines.exceptions.StackTraceRecoveryChannelsTest$sendWithContext$2.invokeSuspend(StackTraceRecoveryChannelsTest.kt:162) at kotlinx.coroutines.exceptions.StackTraceRecoveryChannelsTest$sendFromScope$2.invokeSuspend(StackTraceRecoveryChannelsTest.kt:172) at kotlinx.coroutines.exceptions.StackTraceRecoveryChannelsTest$testSendFromScope$1$deferred$1.invokeSuspend(StackTraceRecoveryChannelsTest.kt:126) Caused by: kotlinx.coroutines.RecoverableTestCancellationException at kotlinx.coroutines.exceptions.StackTraceRecoveryChannelsTest$testSendFromScope$1.invokeSuspend(StackTraceRecoveryChannelsTest.kt:136) - at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
\ No newline at end of file + at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33) diff --git a/kotlinx-coroutines-core/jvm/test-resources/stacktraces/channels/testSendToChannel.txt b/kotlinx-coroutines-core/jvm/test-resources/stacktraces/channels/testSendToChannel.txt index dab728fa..e40cc741 100644 --- a/kotlinx-coroutines-core/jvm/test-resources/stacktraces/channels/testSendToChannel.txt +++ b/kotlinx-coroutines-core/jvm/test-resources/stacktraces/channels/testSendToChannel.txt @@ -1,12 +1,20 @@ -java.util.concurrent.CancellationException: RendezvousChannel was cancelled - at kotlinx.coroutines.channels.AbstractChannel.cancel(AbstractChannel.kt:630) - at kotlinx.coroutines.channels.ReceiveChannel$DefaultImpls.cancel$default(Channel.kt:311) - at kotlinx.coroutines.exceptions.StackTraceRecoveryChannelsTest$testSendToChannel$1$job$1.invokeSuspend(StackTraceRecoveryChannelsTest.kt:52) - (Coroutine boundary) - at kotlinx.coroutines.exceptions.StackTraceRecoveryChannelsTest.channelSend(StackTraceRecoveryChannelsTest.kt:73) - at kotlinx.coroutines.exceptions.StackTraceRecoveryChannelsTest$testSendToChannel$1.invokeSuspend(StackTraceRecoveryChannelsTest.kt:56) -Caused by: java.util.concurrent.CancellationException: RendezvousChannel was cancelled - at kotlinx.coroutines.channels.AbstractChannel.cancel(AbstractChannel.kt:630) - at kotlinx.coroutines.channels.ReceiveChannel$DefaultImpls.cancel$default(Channel.kt:311) - at kotlinx.coroutines.exceptions.StackTraceRecoveryChannelsTest$testSendToChannel$1$job$1.invokeSuspend(StackTraceRecoveryChannelsTest.kt:52) - at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
\ No newline at end of file +java.util.concurrent.CancellationException: Channel was cancelled + at kotlinx.coroutines.channels.BufferedChannel.cancelImpl$(BufferedChannel.kt) + at kotlinx.coroutines.channels.BufferedChannel.cancel(BufferedChannel.kt) + at kotlinx.coroutines.channels.ReceiveChannel$DefaultImpls.cancel$default(Channel.kt) + at kotlinx.coroutines.exceptions.StackTraceRecoveryChannelsTest$testSendToChannel$1$job$1.invokeSuspend(StackTraceRecoveryChannelsTest.kt) + at _COROUTINE._BOUNDARY._(CoroutineDebugging.kt) + at kotlinx.coroutines.exceptions.StackTraceRecoveryChannelsTest.channelSend(StackTraceRecoveryChannelsTest.kt) + at kotlinx.coroutines.exceptions.StackTraceRecoveryChannelsTest$testSendToChannel$1.invokeSuspend(StackTraceRecoveryChannelsTest.kt) +Caused by: java.util.concurrent.CancellationException: Channel was cancelled + at kotlinx.coroutines.channels.BufferedChannel.cancelImpl$(BufferedChannel.kt) + at kotlinx.coroutines.channels.BufferedChannel.cancel(BufferedChannel.kt) + at kotlinx.coroutines.channels.ReceiveChannel$DefaultImpls.cancel$default(Channel.kt) + at kotlinx.coroutines.exceptions.StackTraceRecoveryChannelsTest$testSendToChannel$1$job$1.invokeSuspend(StackTraceRecoveryChannelsTest.kt) + at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt) + at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt) + at kotlinx.coroutines.EventLoopImplBase.processNextEvent(EventLoop.common.kt) + at kotlinx.coroutines.BlockingCoroutine.joinBlocking(Builders.kt) + at kotlinx.coroutines.BuildersKt__BuildersKt.runBlocking(Builders.kt) + at kotlinx.coroutines.BuildersKt.runBlocking(Unknown Source) + at kotlinx.coroutines.TestBase.runTest(TestBase.kt) diff --git a/kotlinx-coroutines-core/jvm/test-resources/stacktraces/channels/testSendToClosedChannel.txt b/kotlinx-coroutines-core/jvm/test-resources/stacktraces/channels/testSendToClosedChannel.txt index 54fdbb32..f2609594 100644 --- a/kotlinx-coroutines-core/jvm/test-resources/stacktraces/channels/testSendToClosedChannel.txt +++ b/kotlinx-coroutines-core/jvm/test-resources/stacktraces/channels/testSendToClosedChannel.txt @@ -1,8 +1,8 @@ kotlinx.coroutines.RecoverableTestException at kotlinx.coroutines.exceptions.StackTraceRecoveryChannelsTest$testSendToClosedChannel$1.invokeSuspend(StackTraceRecoveryChannelsTest.kt:43) - (Coroutine boundary) + at _COROUTINE._BOUNDARY._(CoroutineDebugging.kt) at kotlinx.coroutines.exceptions.StackTraceRecoveryChannelsTest.channelSend(StackTraceRecoveryChannelsTest.kt:74) at kotlinx.coroutines.exceptions.StackTraceRecoveryChannelsTest$testSendToClosedChannel$1.invokeSuspend(StackTraceRecoveryChannelsTest.kt:44) Caused by: kotlinx.coroutines.RecoverableTestException at kotlinx.coroutines.exceptions.StackTraceRecoveryChannelsTest$testSendToClosedChannel$1.invokeSuspend(StackTraceRecoveryChannelsTest.kt:43) - at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
\ No newline at end of file + at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33) diff --git a/kotlinx-coroutines-core/jvm/test-resources/stacktraces/resume-mode/testEventLoopDispatcher.txt b/kotlinx-coroutines-core/jvm/test-resources/stacktraces/resume-mode/testEventLoopDispatcher.txt index 6b40ec83..0e75e645 100644 --- a/kotlinx-coroutines-core/jvm/test-resources/stacktraces/resume-mode/testEventLoopDispatcher.txt +++ b/kotlinx-coroutines-core/jvm/test-resources/stacktraces/resume-mode/testEventLoopDispatcher.txt @@ -1,7 +1,7 @@ kotlinx.coroutines.RecoverableTestException at kotlinx.coroutines.exceptions.StackTraceRecoveryResumeModeTest.testResumeModeFastPath(StackTraceRecoveryResumeModeTest.kt:61) at kotlinx.coroutines.exceptions.StackTraceRecoveryResumeModeTest$testEventLoopDispatcher$1.invokeSuspend(StackTraceRecoveryResumeModeTest.kt:40) - (Coroutine boundary) + at _COROUTINE._BOUNDARY._(CoroutineDebugging.kt) at kotlinx.coroutines.exceptions.StackTraceRecoveryResumeModeTest$withContext$2.invokeSuspend(StackTraceRecoveryResumeModeTest.kt:76) at kotlinx.coroutines.exceptions.StackTraceRecoveryResumeModeTest.doFastPath(StackTraceRecoveryResumeModeTest.kt:71) at kotlinx.coroutines.exceptions.StackTraceRecoveryResumeModeTest.testResumeModeFastPath(StackTraceRecoveryResumeModeTest.kt:62) @@ -9,4 +9,4 @@ kotlinx.coroutines.RecoverableTestException Caused by: kotlinx.coroutines.RecoverableTestException at kotlinx.coroutines.exceptions.StackTraceRecoveryResumeModeTest.testResumeModeFastPath(StackTraceRecoveryResumeModeTest.kt:61) at kotlinx.coroutines.exceptions.StackTraceRecoveryResumeModeTest$testEventLoopDispatcher$1.invokeSuspend(StackTraceRecoveryResumeModeTest.kt:40) - at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
\ No newline at end of file + at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33) diff --git a/kotlinx-coroutines-core/jvm/test-resources/stacktraces/resume-mode/testEventLoopDispatcherSuspending.txt b/kotlinx-coroutines-core/jvm/test-resources/stacktraces/resume-mode/testEventLoopDispatcherSuspending.txt index 5afc559f..0792646e 100644 --- a/kotlinx-coroutines-core/jvm/test-resources/stacktraces/resume-mode/testEventLoopDispatcherSuspending.txt +++ b/kotlinx-coroutines-core/jvm/test-resources/stacktraces/resume-mode/testEventLoopDispatcherSuspending.txt @@ -1,10 +1,10 @@ otlinx.coroutines.RecoverableTestException at kotlinx.coroutines.exceptions.StackTraceRecoveryResumeModeTest$testResumeModeSuspending$2.invokeSuspend(StackTraceRecoveryResumeModeTest.kt:99) - (Coroutine boundary) + at _COROUTINE._BOUNDARY._(CoroutineDebugging.kt) at kotlinx.coroutines.exceptions.StackTraceRecoveryResumeModeTest$withContext$4.invokeSuspend(StackTraceRecoveryResumeModeTest.kt:116) at kotlinx.coroutines.exceptions.StackTraceRecoveryResumeModeTest.doSuspendingPath(StackTraceRecoveryResumeModeTest.kt:110) at kotlinx.coroutines.exceptions.StackTraceRecoveryResumeModeTest.testResumeModeSuspending(StackTraceRecoveryResumeModeTest.kt:101) at kotlinx.coroutines.exceptions.StackTraceRecoveryResumeModeTest$testEventLoopDispatcherSuspending$1.invokeSuspend(StackTraceRecoveryResumeModeTest.kt:89) Caused by: kotlinx.coroutines.RecoverableTestException at kotlinx.coroutines.exceptions.StackTraceRecoveryResumeModeTest$testResumeModeSuspending$2.invokeSuspend(StackTraceRecoveryResumeModeTest.kt:99) - at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
\ No newline at end of file + at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33) diff --git a/kotlinx-coroutines-core/jvm/test-resources/stacktraces/resume-mode/testNestedEventLoopChangedContext.txt b/kotlinx-coroutines-core/jvm/test-resources/stacktraces/resume-mode/testNestedEventLoopChangedContext.txt index 406b2d1c..f3ca1fc4 100644 --- a/kotlinx-coroutines-core/jvm/test-resources/stacktraces/resume-mode/testNestedEventLoopChangedContext.txt +++ b/kotlinx-coroutines-core/jvm/test-resources/stacktraces/resume-mode/testNestedEventLoopChangedContext.txt @@ -1,7 +1,7 @@ kotlinx.coroutines.RecoverableTestException at kotlinx.coroutines.exceptions.StackTraceRecoveryResumeModeTest.testResumeModeFastPath(StackTraceRecoveryResumeModeTest.kt:61) at kotlinx.coroutines.exceptions.StackTraceRecoveryResumeModeTest$testNestedEventLoopChangedContext$1$1.invokeSuspend(StackTraceRecoveryResumeModeTest.kt:54) - (Coroutine boundary) + at _COROUTINE._BOUNDARY._(CoroutineDebugging.kt) at kotlinx.coroutines.exceptions.StackTraceRecoveryResumeModeTest$withContext$2.invokeSuspend(StackTraceRecoveryResumeModeTest.kt:76) at kotlinx.coroutines.exceptions.StackTraceRecoveryResumeModeTest.doFastPath(StackTraceRecoveryResumeModeTest.kt:71) at kotlinx.coroutines.exceptions.StackTraceRecoveryResumeModeTest.testResumeModeFastPath(StackTraceRecoveryResumeModeTest.kt:62) diff --git a/kotlinx-coroutines-core/jvm/test-resources/stacktraces/resume-mode/testNestedEventLoopChangedContextSuspending.txt b/kotlinx-coroutines-core/jvm/test-resources/stacktraces/resume-mode/testNestedEventLoopChangedContextSuspending.txt index 86ec5e4b..dbb574fe 100644 --- a/kotlinx-coroutines-core/jvm/test-resources/stacktraces/resume-mode/testNestedEventLoopChangedContextSuspending.txt +++ b/kotlinx-coroutines-core/jvm/test-resources/stacktraces/resume-mode/testNestedEventLoopChangedContextSuspending.txt @@ -1,6 +1,6 @@ kotlinx.coroutines.RecoverableTestException at kotlinx.coroutines.exceptions.StackTraceRecoveryResumeModeTest$testResumeModeSuspending$2.invokeSuspend(StackTraceRecoveryResumeModeTest.kt:113) - (Coroutine boundary) + at _COROUTINE._BOUNDARY._(CoroutineDebugging.kt) at kotlinx.coroutines.exceptions.StackTraceRecoveryResumeModeTest$withContext$4.invokeSuspend(StackTraceRecoveryResumeModeTest.kt:130) at kotlinx.coroutines.exceptions.StackTraceRecoveryResumeModeTest.doSuspendingPath(StackTraceRecoveryResumeModeTest.kt:124) at kotlinx.coroutines.exceptions.StackTraceRecoveryResumeModeTest.testResumeModeSuspending(StackTraceRecoveryResumeModeTest.kt:115) @@ -8,4 +8,4 @@ kotlinx.coroutines.RecoverableTestException at kotlinx.coroutines.exceptions.StackTraceRecoveryResumeModeTest$testNestedEventLoopChangedContextSuspending$1.invokeSuspend(StackTraceRecoveryResumeModeTest.kt:102) Caused by: kotlinx.coroutines.RecoverableTestException at kotlinx.coroutines.exceptions.StackTraceRecoveryResumeModeTest$testResumeModeSuspending$2.invokeSuspend(StackTraceRecoveryResumeModeTest.kt:113) - at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
\ No newline at end of file + at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33) diff --git a/kotlinx-coroutines-core/jvm/test-resources/stacktraces/resume-mode/testNestedEventLoopDispatcher.txt b/kotlinx-coroutines-core/jvm/test-resources/stacktraces/resume-mode/testNestedEventLoopDispatcher.txt index d9098bba..e17e2db6 100644 --- a/kotlinx-coroutines-core/jvm/test-resources/stacktraces/resume-mode/testNestedEventLoopDispatcher.txt +++ b/kotlinx-coroutines-core/jvm/test-resources/stacktraces/resume-mode/testNestedEventLoopDispatcher.txt @@ -1,7 +1,7 @@ kotlinx.coroutines.RecoverableTestException at kotlinx.coroutines.exceptions.StackTraceRecoveryResumeModeTest.testResumeModeFastPath(StackTraceRecoveryResumeModeTest.kt:61) at kotlinx.coroutines.exceptions.StackTraceRecoveryResumeModeTest$testNestedEventLoopDispatcher$1$1.invokeSuspend(StackTraceRecoveryResumeModeTest.kt:47) - (Coroutine boundary) + at _COROUTINE._BOUNDARY._(CoroutineDebugging.kt) at kotlinx.coroutines.exceptions.StackTraceRecoveryResumeModeTest$withContext$2.invokeSuspend(StackTraceRecoveryResumeModeTest.kt:76) at kotlinx.coroutines.exceptions.StackTraceRecoveryResumeModeTest.doFastPath(StackTraceRecoveryResumeModeTest.kt:71) at kotlinx.coroutines.exceptions.StackTraceRecoveryResumeModeTest.testResumeModeFastPath(StackTraceRecoveryResumeModeTest.kt:62) diff --git a/kotlinx-coroutines-core/jvm/test-resources/stacktraces/resume-mode/testNestedEventLoopDispatcherSuspending.txt b/kotlinx-coroutines-core/jvm/test-resources/stacktraces/resume-mode/testNestedEventLoopDispatcherSuspending.txt index 8caed7ac..26e03599 100644 --- a/kotlinx-coroutines-core/jvm/test-resources/stacktraces/resume-mode/testNestedEventLoopDispatcherSuspending.txt +++ b/kotlinx-coroutines-core/jvm/test-resources/stacktraces/resume-mode/testNestedEventLoopDispatcherSuspending.txt @@ -1,6 +1,6 @@ kotlinx.coroutines.RecoverableTestException at kotlinx.coroutines.exceptions.StackTraceRecoveryResumeModeTest$testResumeModeSuspending$2.invokeSuspend(StackTraceRecoveryResumeModeTest.kt:113) - (Coroutine boundary) + at _COROUTINE._BOUNDARY._(CoroutineDebugging.kt) at kotlinx.coroutines.exceptions.StackTraceRecoveryResumeModeTest$withContext$4.invokeSuspend(StackTraceRecoveryResumeModeTest.kt:130) at kotlinx.coroutines.exceptions.StackTraceRecoveryResumeModeTest.doSuspendingPath(StackTraceRecoveryResumeModeTest.kt:124) at kotlinx.coroutines.exceptions.StackTraceRecoveryResumeModeTest.testResumeModeSuspending(StackTraceRecoveryResumeModeTest.kt:115) @@ -8,4 +8,4 @@ kotlinx.coroutines.RecoverableTestException at kotlinx.coroutines.exceptions.StackTraceRecoveryResumeModeTest$testNestedEventLoopDispatcherSuspending$1.invokeSuspend(StackTraceRecoveryResumeModeTest.kt:95) Caused by: kotlinx.coroutines.RecoverableTestException at kotlinx.coroutines.exceptions.StackTraceRecoveryResumeModeTest$testResumeModeSuspending$2.invokeSuspend(StackTraceRecoveryResumeModeTest.kt:113) - at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
\ No newline at end of file + at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33) diff --git a/kotlinx-coroutines-core/jvm/test-resources/stacktraces/resume-mode/testNestedUnconfined.txt b/kotlinx-coroutines-core/jvm/test-resources/stacktraces/resume-mode/testNestedUnconfined.txt index a2cd009d..f247920e 100644 --- a/kotlinx-coroutines-core/jvm/test-resources/stacktraces/resume-mode/testNestedUnconfined.txt +++ b/kotlinx-coroutines-core/jvm/test-resources/stacktraces/resume-mode/testNestedUnconfined.txt @@ -1,7 +1,7 @@ kotlinx.coroutines.RecoverableTestException at kotlinx.coroutines.exceptions.StackTraceRecoveryResumeModeTest.testResumeModeFastPath(StackTraceRecoveryResumeModeTest.kt:61) at kotlinx.coroutines.exceptions.StackTraceRecoveryResumeModeTest$testNestedUnconfined$1$1.invokeSuspend(StackTraceRecoveryResumeModeTest.kt:27) - (Coroutine boundary) + at _COROUTINE._BOUNDARY._(CoroutineDebugging.kt) at kotlinx.coroutines.exceptions.StackTraceRecoveryResumeModeTest$withContext$2.invokeSuspend(StackTraceRecoveryResumeModeTest.kt:76) at kotlinx.coroutines.exceptions.StackTraceRecoveryResumeModeTest.doFastPath(StackTraceRecoveryResumeModeTest.kt:71) at kotlinx.coroutines.exceptions.StackTraceRecoveryResumeModeTest.testResumeModeFastPath(StackTraceRecoveryResumeModeTest.kt:62) @@ -10,4 +10,4 @@ kotlinx.coroutines.RecoverableTestException Caused by: kotlinx.coroutines.RecoverableTestException at kotlinx.coroutines.exceptions.StackTraceRecoveryResumeModeTest.testResumeModeFastPath(StackTraceRecoveryResumeModeTest.kt:61) at kotlinx.coroutines.exceptions.StackTraceRecoveryResumeModeTest$testNestedUnconfined$1$1.invokeSuspend(StackTraceRecoveryResumeModeTest.kt:27) - at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
\ No newline at end of file + at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33) diff --git a/kotlinx-coroutines-core/jvm/test-resources/stacktraces/resume-mode/testNestedUnconfinedChangedContext.txt b/kotlinx-coroutines-core/jvm/test-resources/stacktraces/resume-mode/testNestedUnconfinedChangedContext.txt index a786682b..b7ae52c9 100644 --- a/kotlinx-coroutines-core/jvm/test-resources/stacktraces/resume-mode/testNestedUnconfinedChangedContext.txt +++ b/kotlinx-coroutines-core/jvm/test-resources/stacktraces/resume-mode/testNestedUnconfinedChangedContext.txt @@ -1,7 +1,7 @@ kotlinx.coroutines.RecoverableTestException at kotlinx.coroutines.exceptions.StackTraceRecoveryResumeModeTest.testResumeModeFastPath(StackTraceRecoveryResumeModeTest.kt:61) at kotlinx.coroutines.exceptions.StackTraceRecoveryResumeModeTest$testNestedUnconfinedChangedContext$1$1.invokeSuspend(StackTraceRecoveryResumeModeTest.kt:34) - (Coroutine boundary) + at _COROUTINE._BOUNDARY._(CoroutineDebugging.kt) at kotlinx.coroutines.exceptions.StackTraceRecoveryResumeModeTest$withContext$2.invokeSuspend(StackTraceRecoveryResumeModeTest.kt:76) at kotlinx.coroutines.exceptions.StackTraceRecoveryResumeModeTest.doFastPath(StackTraceRecoveryResumeModeTest.kt:71) at kotlinx.coroutines.exceptions.StackTraceRecoveryResumeModeTest.testResumeModeFastPath(StackTraceRecoveryResumeModeTest.kt:62) diff --git a/kotlinx-coroutines-core/jvm/test-resources/stacktraces/resume-mode/testNestedUnconfinedChangedContextSuspending.txt b/kotlinx-coroutines-core/jvm/test-resources/stacktraces/resume-mode/testNestedUnconfinedChangedContextSuspending.txt index 8c937a7c..241a3b23 100644 --- a/kotlinx-coroutines-core/jvm/test-resources/stacktraces/resume-mode/testNestedUnconfinedChangedContextSuspending.txt +++ b/kotlinx-coroutines-core/jvm/test-resources/stacktraces/resume-mode/testNestedUnconfinedChangedContextSuspending.txt @@ -1,6 +1,6 @@ kotlinx.coroutines.RecoverableTestException at kotlinx.coroutines.exceptions.StackTraceRecoveryResumeModeTest$testResumeModeSuspending$2.invokeSuspend(StackTraceRecoveryResumeModeTest.kt:128) - (Coroutine boundary) + at _COROUTINE._BOUNDARY._(CoroutineDebugging.kt) at kotlinx.coroutines.exceptions.StackTraceRecoveryResumeModeTest$withContext$4.invokeSuspend(StackTraceRecoveryResumeModeTest.kt:148) at kotlinx.coroutines.exceptions.StackTraceRecoveryResumeModeTest.doSuspendingPath(StackTraceRecoveryResumeModeTest.kt:140) at kotlinx.coroutines.exceptions.StackTraceRecoveryResumeModeTest.testResumeModeSuspending(StackTraceRecoveryResumeModeTest.kt:130) @@ -8,4 +8,4 @@ kotlinx.coroutines.RecoverableTestException at kotlinx.coroutines.exceptions.StackTraceRecoveryResumeModeTest$testNestedUnconfinedChangedContextSuspending$1.invokeSuspend(StackTraceRecoveryResumeModeTest.kt:94) Caused by: kotlinx.coroutines.RecoverableTestException at kotlinx.coroutines.exceptions.StackTraceRecoveryResumeModeTest$testResumeModeSuspending$2.invokeSuspend(StackTraceRecoveryResumeModeTest.kt:128) - at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
\ No newline at end of file + at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33) diff --git a/kotlinx-coroutines-core/jvm/test-resources/stacktraces/resume-mode/testNestedUnconfinedSuspending.txt b/kotlinx-coroutines-core/jvm/test-resources/stacktraces/resume-mode/testNestedUnconfinedSuspending.txt index b6eef479..4484c664 100644 --- a/kotlinx-coroutines-core/jvm/test-resources/stacktraces/resume-mode/testNestedUnconfinedSuspending.txt +++ b/kotlinx-coroutines-core/jvm/test-resources/stacktraces/resume-mode/testNestedUnconfinedSuspending.txt @@ -1,6 +1,6 @@ kotlinx.coroutines.RecoverableTestException at kotlinx.coroutines.exceptions.StackTraceRecoveryResumeModeTest$testResumeModeSuspending$2.invokeSuspend(StackTraceRecoveryResumeModeTest.kt:128) - (Coroutine boundary) + at _COROUTINE._BOUNDARY._(CoroutineDebugging.kt) at kotlinx.coroutines.exceptions.StackTraceRecoveryResumeModeTest$withContext$4.invokeSuspend(StackTraceRecoveryResumeModeTest.kt:148) at kotlinx.coroutines.exceptions.StackTraceRecoveryResumeModeTest.doSuspendingPath(StackTraceRecoveryResumeModeTest.kt:140) at kotlinx.coroutines.exceptions.StackTraceRecoveryResumeModeTest.testResumeModeSuspending(StackTraceRecoveryResumeModeTest.kt:130) @@ -8,4 +8,4 @@ kotlinx.coroutines.RecoverableTestException at kotlinx.coroutines.exceptions.StackTraceRecoveryResumeModeTest$testNestedUnconfinedSuspending$1.invokeSuspend(StackTraceRecoveryResumeModeTest.kt:87) Caused by: kotlinx.coroutines.RecoverableTestException at kotlinx.coroutines.exceptions.StackTraceRecoveryResumeModeTest$testResumeModeSuspending$2.invokeSuspend(StackTraceRecoveryResumeModeTest.kt:128) - at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
\ No newline at end of file + at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33) diff --git a/kotlinx-coroutines-core/jvm/test-resources/stacktraces/resume-mode/testUnconfined.txt b/kotlinx-coroutines-core/jvm/test-resources/stacktraces/resume-mode/testUnconfined.txt index 9b9cba3e..9fc71678 100644 --- a/kotlinx-coroutines-core/jvm/test-resources/stacktraces/resume-mode/testUnconfined.txt +++ b/kotlinx-coroutines-core/jvm/test-resources/stacktraces/resume-mode/testUnconfined.txt @@ -1,12 +1,9 @@ kotlinx.coroutines.RecoverableTestException - at kotlinx.coroutines.exceptions.StackTraceRecoveryResumeModeTest.testResumeModeFastPath(StackTraceRecoveryResumeModeTest.kt:61) - at kotlinx.coroutines.exceptions.StackTraceRecoveryResumeModeTest$testUnconfined$1.invokeSuspend(StackTraceRecoveryResumeModeTest.kt:40) - (Coroutine boundary) - at kotlinx.coroutines.exceptions.StackTraceRecoveryResumeModeTest$withContext$2.invokeSuspend(StackTraceRecoveryResumeModeTest.kt:76) - at kotlinx.coroutines.exceptions.StackTraceRecoveryResumeModeTest.doFastPath(StackTraceRecoveryResumeModeTest.kt:71) - at kotlinx.coroutines.exceptions.StackTraceRecoveryResumeModeTest.testResumeModeFastPath(StackTraceRecoveryResumeModeTest.kt:62) - at kotlinx.coroutines.exceptions.StackTraceRecoveryResumeModeTest$testUnconfined$1.invokeSuspend(StackTraceRecoveryResumeModeTest.kt:40) + at kotlinx.coroutines.channels.BufferedChannel.receive$suspendImpl(BufferedChannel.kt) + at kotlinx.coroutines.channels.BufferedChannel.receive(BufferedChannel.kt) + at kotlinx.coroutines.exceptions.StackTraceRecoveryResumeModeTest$withContext$2.invokeSuspend(StackTraceRecoveryResumeModeTest.kt) Caused by: kotlinx.coroutines.RecoverableTestException - at kotlinx.coroutines.exceptions.StackTraceRecoveryResumeModeTest.testResumeModeFastPath(StackTraceRecoveryResumeModeTest.kt:61) - at kotlinx.coroutines.exceptions.StackTraceRecoveryResumeModeTest$testUnconfined$1.invokeSuspend(StackTraceRecoveryResumeModeTest.kt:40) - at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
\ No newline at end of file + at kotlinx.coroutines.exceptions.StackTraceRecoveryResumeModeTest.testResumeModeFastPath(StackTraceRecoveryResumeModeTest.kt) + at kotlinx.coroutines.exceptions.StackTraceRecoveryResumeModeTest.access$testResumeModeFastPath(StackTraceRecoveryResumeModeTest.kt) + at kotlinx.coroutines.exceptions.StackTraceRecoveryResumeModeTest$testUnconfined$1.invokeSuspend(StackTraceRecoveryResumeModeTest.kt) + at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt) diff --git a/kotlinx-coroutines-core/jvm/test-resources/stacktraces/resume-mode/testUnconfinedSuspending.txt b/kotlinx-coroutines-core/jvm/test-resources/stacktraces/resume-mode/testUnconfinedSuspending.txt index ca0bbe7f..fb742a30 100644 --- a/kotlinx-coroutines-core/jvm/test-resources/stacktraces/resume-mode/testUnconfinedSuspending.txt +++ b/kotlinx-coroutines-core/jvm/test-resources/stacktraces/resume-mode/testUnconfinedSuspending.txt @@ -1,9 +1,9 @@ kotlinx.coroutines.RecoverableTestException at kotlinx.coroutines.exceptions.StackTraceRecoveryResumeModeTest$testResumeModeSuspending$2.invokeSuspend(StackTraceRecoveryResumeModeTest.kt:128) - (Coroutine boundary) + at _COROUTINE._BOUNDARY._(CoroutineDebugging.kt) at kotlinx.coroutines.exceptions.StackTraceRecoveryResumeModeTest.doSuspendingPath(StackTraceRecoveryResumeModeTest.kt:140) at kotlinx.coroutines.exceptions.StackTraceRecoveryResumeModeTest.testResumeModeSuspending(StackTraceRecoveryResumeModeTest.kt:130) at kotlinx.coroutines.exceptions.StackTraceRecoveryResumeModeTest$testUnconfinedSuspending$1.invokeSuspend(StackTraceRecoveryResumeModeTest.kt:82) Caused by: kotlinx.coroutines.RecoverableTestException at kotlinx.coroutines.exceptions.StackTraceRecoveryResumeModeTest$testResumeModeSuspending$2.invokeSuspend(StackTraceRecoveryResumeModeTest.kt:128) - at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
\ No newline at end of file + at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33) diff --git a/kotlinx-coroutines-core/jvm/test-resources/stacktraces/select/testSelectCompletedAwait.txt b/kotlinx-coroutines-core/jvm/test-resources/stacktraces/select/testSelectCompletedAwait.txt index dbc39ccc..2e86a7ad 100644 --- a/kotlinx-coroutines-core/jvm/test-resources/stacktraces/select/testSelectCompletedAwait.txt +++ b/kotlinx-coroutines-core/jvm/test-resources/stacktraces/select/testSelectCompletedAwait.txt @@ -1,6 +1,6 @@ kotlinx.coroutines.RecoverableTestException at kotlinx.coroutines.exceptions.StackTraceRecoverySelectTest$testSelectCompletedAwait$1.invokeSuspend(StackTraceRecoverySelectTest.kt:40) - (Coroutine boundary) + at _COROUTINE._BOUNDARY._(CoroutineDebugging.kt) at kotlinx.coroutines.exceptions.StackTraceRecoverySelectTest$testSelectCompletedAwait$1.invokeSuspend(StackTraceRecoverySelectTest.kt:41) Caused by: kotlinx.coroutines.RecoverableTestException at kotlinx.coroutines.exceptions.StackTraceRecoverySelectTest$testSelectCompletedAwait$1.invokeSuspend(StackTraceRecoverySelectTest.kt:40) diff --git a/kotlinx-coroutines-core/jvm/test-resources/stacktraces/select/testSelectJoin.txt b/kotlinx-coroutines-core/jvm/test-resources/stacktraces/select/testSelectJoin.txt index 3bfd08e5..420aa7e9 100644 --- a/kotlinx-coroutines-core/jvm/test-resources/stacktraces/select/testSelectJoin.txt +++ b/kotlinx-coroutines-core/jvm/test-resources/stacktraces/select/testSelectJoin.txt @@ -1,6 +1,7 @@ kotlinx.coroutines.RecoverableTestException at kotlinx.coroutines.exceptions.StackTraceRecoverySelectTest$doSelect$2$1.invokeSuspend(StackTraceRecoverySelectTest.kt) - (Coroutine boundary) + at _COROUTINE._BOUNDARY._(CoroutineDebugging.kt) + at kotlinx.coroutines.selects.SelectImplementation.processResultAndInvokeBlockRecoveringException(Select.kt) at kotlinx.coroutines.exceptions.StackTraceRecoverySelectTest$testSelectJoin$1.invokeSuspend(StackTraceRecoverySelectTest.kt) Caused by: kotlinx.coroutines.RecoverableTestException at kotlinx.coroutines.exceptions.StackTraceRecoverySelectTest$doSelect$2$1.invokeSuspend(StackTraceRecoverySelectTest.kt) diff --git a/kotlinx-coroutines-core/jvm/test-resources/stacktraces/select/testSelectOnReceive.txt b/kotlinx-coroutines-core/jvm/test-resources/stacktraces/select/testSelectOnReceive.txt new file mode 100644 index 00000000..8b958d20 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test-resources/stacktraces/select/testSelectOnReceive.txt @@ -0,0 +1,32 @@ +kotlinx.coroutines.channels.ClosedReceiveChannelException: Channel was closed + at kotlinx.coroutines.channels.BufferedChannel.getReceiveException(BufferedChannel.kt) + at kotlinx.coroutines.channels.BufferedChannel.processResultSelectReceive(BufferedChannel.kt) + at kotlinx.coroutines.channels.BufferedChannel.access$processResultSelectReceive(BufferedChannel.kt) + at kotlinx.coroutines.channels.BufferedChannel$onReceive$2.invoke(BufferedChannel.kt) + at kotlinx.coroutines.channels.BufferedChannel$onReceive$2.invoke(BufferedChannel.kt) + at kotlinx.coroutines.selects.SelectImplementation$ClauseData.processResult(Select.kt) + at kotlinx.coroutines.selects.SelectImplementation.processResultAndInvokeBlockRecoveringException(Select.kt) + at kotlinx.coroutines.selects.SelectImplementation.complete(Select.kt) + at kotlinx.coroutines.selects.SelectImplementation.doSelect$suspendImpl(Select.kt) + at kotlinx.coroutines.selects.SelectImplementation.doSelect(Select.kt) + at kotlinx.coroutines.exceptions.StackTraceRecoverySelectTest.doSelectOnReceive(StackTraceRecoverySelectTest.kt) + at kotlinx.coroutines.exceptions.StackTraceRecoverySelectTest.access$doSelectOnReceive(StackTraceRecoverySelectTest.kt) + at kotlinx.coroutines.exceptions.StackTraceRecoverySelectTest$testSelectOnReceive$1.invokeSuspend(StackTraceRecoverySelectTest.kt) + at _COROUTINE._BOUNDARY._(CoroutineDebugging.kt) + at kotlinx.coroutines.selects.SelectImplementation.processResultAndInvokeBlockRecoveringException(Select.kt) + at kotlinx.coroutines.exceptions.StackTraceRecoverySelectTest$testSelectOnReceive$1.invokeSuspend(StackTraceRecoverySelectTest.kt) +Caused by: kotlinx.coroutines.channels.ClosedReceiveChannelException: Channel was closed + at kotlinx.coroutines.channels.BufferedChannel.getReceiveException(BufferedChannel.kt) + at kotlinx.coroutines.channels.BufferedChannel.processResultSelectReceive(BufferedChannel.kt) + at kotlinx.coroutines.channels.BufferedChannel.access$processResultSelectReceive(BufferedChannel.kt) + at kotlinx.coroutines.channels.BufferedChannel$onReceive$2.invoke(BufferedChannel.kt) + at kotlinx.coroutines.channels.BufferedChannel$onReceive$2.invoke(BufferedChannel.kt) + at kotlinx.coroutines.selects.SelectImplementation$ClauseData.processResult(Select.kt) + at kotlinx.coroutines.selects.SelectImplementation.processResultAndInvokeBlockRecoveringException(Select.kt) + at kotlinx.coroutines.selects.SelectImplementation.complete(Select.kt) + at kotlinx.coroutines.selects.SelectImplementation.doSelect$suspendImpl(Select.kt) + at kotlinx.coroutines.selects.SelectImplementation.doSelect(Select.kt) + at kotlinx.coroutines.exceptions.StackTraceRecoverySelectTest.doSelectOnReceive(StackTraceRecoverySelectTest.kt) + at kotlinx.coroutines.exceptions.StackTraceRecoverySelectTest.access$doSelectOnReceive(StackTraceRecoverySelectTest.kt) + at kotlinx.coroutines.exceptions.StackTraceRecoverySelectTest$testSelectOnReceive$1.invokeSuspend(StackTraceRecoverySelectTest.kt) + at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt) diff --git a/kotlinx-coroutines-core/jvm/test-resources/stacktraces/timeout/testStacktraceIsRecoveredFromLexicalBlockWhenTriggeredByChild.txt b/kotlinx-coroutines-core/jvm/test-resources/stacktraces/timeout/testStacktraceIsRecoveredFromLexicalBlockWhenTriggeredByChild.txt index ab23c9a3..ac40dc15 100644 --- a/kotlinx-coroutines-core/jvm/test-resources/stacktraces/timeout/testStacktraceIsRecoveredFromLexicalBlockWhenTriggeredByChild.txt +++ b/kotlinx-coroutines-core/jvm/test-resources/stacktraces/timeout/testStacktraceIsRecoveredFromLexicalBlockWhenTriggeredByChild.txt @@ -1,7 +1,7 @@ kotlinx.coroutines.TimeoutCancellationException: Timed out waiting for 200 ms - (Coroutine boundary) + at _COROUTINE._BOUNDARY._(CoroutineDebugging.kt) at kotlinx.coroutines.exceptions.StackTraceRecoveryWithTimeoutTest.outerChildWithTimeout(StackTraceRecoveryWithTimeoutTest.kt:48) at kotlinx.coroutines.exceptions.StackTraceRecoveryWithTimeoutTest$testStacktraceIsRecoveredFromLexicalBlockWhenTriggeredByChild$1.invokeSuspend(StackTraceRecoveryWithTimeoutTest.kt:40) Caused by: kotlinx.coroutines.TimeoutCancellationException: Timed out waiting for 200 ms at kotlinx.coroutines.TimeoutKt.TimeoutCancellationException(Timeout.kt:116) - at kotlinx.coroutines.TimeoutCoroutine.run(Timeout.kt:86)
\ No newline at end of file + at kotlinx.coroutines.TimeoutCoroutine.run(Timeout.kt:86) diff --git a/kotlinx-coroutines-core/jvm/test-resources/stacktraces/timeout/testStacktraceIsRecoveredFromSuspensionPoint.txt b/kotlinx-coroutines-core/jvm/test-resources/stacktraces/timeout/testStacktraceIsRecoveredFromSuspensionPoint.txt index d3497fac..9d5ddb66 100644 --- a/kotlinx-coroutines-core/jvm/test-resources/stacktraces/timeout/testStacktraceIsRecoveredFromSuspensionPoint.txt +++ b/kotlinx-coroutines-core/jvm/test-resources/stacktraces/timeout/testStacktraceIsRecoveredFromSuspensionPoint.txt @@ -1,5 +1,5 @@ kotlinx.coroutines.TimeoutCancellationException: Timed out waiting for 200 ms - (Coroutine boundary) + at _COROUTINE._BOUNDARY._(CoroutineDebugging.kt) at kotlinx.coroutines.exceptions.StackTraceRecoveryWithTimeoutTest.suspendForever(StackTraceRecoveryWithTimeoutTest.kt:42) at kotlinx.coroutines.exceptions.StackTraceRecoveryWithTimeoutTest$outerWithTimeout$2.invokeSuspend(StackTraceRecoveryWithTimeoutTest.kt:32) at kotlinx.coroutines.exceptions.StackTraceRecoveryWithTimeoutTest.outerWithTimeout(StackTraceRecoveryWithTimeoutTest.kt:31) @@ -7,4 +7,4 @@ kotlinx.coroutines.TimeoutCancellationException: Timed out waiting for 200 ms Caused by: kotlinx.coroutines.TimeoutCancellationException: Timed out waiting for 200 ms at kotlinx.coroutines.TimeoutKt.TimeoutCancellationException(Timeout.kt:116) at kotlinx.coroutines.TimeoutCoroutine.run(Timeout.kt:86) - at kotlinx.coroutines.EventLoopImplBase$DelayedRunnableTask.run(EventLoop.common.kt:492)
\ No newline at end of file + at kotlinx.coroutines.EventLoopImplBase$DelayedRunnableTask.run(EventLoop.common.kt:492) diff --git a/kotlinx-coroutines-core/jvm/test-resources/stacktraces/timeout/testStacktraceIsRecoveredFromSuspensionPointWithChild.txt b/kotlinx-coroutines-core/jvm/test-resources/stacktraces/timeout/testStacktraceIsRecoveredFromSuspensionPointWithChild.txt index 8ec7691e..6f21cc6b 100644 --- a/kotlinx-coroutines-core/jvm/test-resources/stacktraces/timeout/testStacktraceIsRecoveredFromSuspensionPointWithChild.txt +++ b/kotlinx-coroutines-core/jvm/test-resources/stacktraces/timeout/testStacktraceIsRecoveredFromSuspensionPointWithChild.txt @@ -1,9 +1,9 @@ kotlinx.coroutines.TimeoutCancellationException: Timed out waiting for 200 ms - (Coroutine boundary) + at _COROUTINE._BOUNDARY._(CoroutineDebugging.kt) at kotlinx.coroutines.exceptions.StackTraceRecoveryWithTimeoutTest.suspendForever(StackTraceRecoveryWithTimeoutTest.kt:92) at kotlinx.coroutines.exceptions.StackTraceRecoveryWithTimeoutTest$outerChild$2.invokeSuspend(StackTraceRecoveryWithTimeoutTest.kt:78) at kotlinx.coroutines.exceptions.StackTraceRecoveryWithTimeoutTest.outerChild(StackTraceRecoveryWithTimeoutTest.kt:74) at kotlinx.coroutines.exceptions.StackTraceRecoveryWithTimeoutTest$testStacktraceIsRecoveredFromSuspensionPointWithChild$1.invokeSuspend(StackTraceRecoveryWithTimeoutTest.kt:66) Caused by: kotlinx.coroutines.TimeoutCancellationException: Timed out waiting for 200 ms at kotlinx.coroutines.TimeoutKt.TimeoutCancellationException(Timeout.kt:116) - at kotlinx.coroutines.TimeoutCoroutine.run(Timeout.kt:86)
\ No newline at end of file + at kotlinx.coroutines.TimeoutCoroutine.run(Timeout.kt:86) diff --git a/kotlinx-coroutines-core/jvm/test/AbstractLincheckTest.kt b/kotlinx-coroutines-core/jvm/test/AbstractLincheckTest.kt index 89bbbfd7..da73ca62 100644 --- a/kotlinx-coroutines-core/jvm/test/AbstractLincheckTest.kt +++ b/kotlinx-coroutines-core/jvm/test/AbstractLincheckTest.kt @@ -9,14 +9,14 @@ import org.jetbrains.kotlinx.lincheck.strategy.stress.* import org.jetbrains.kotlinx.lincheck.verifier.* import org.junit.* -abstract class AbstractLincheckTest : VerifierState() { +abstract class AbstractLincheckTest { open fun <O: Options<O, *>> O.customize(isStressTest: Boolean): O = this open fun ModelCheckingOptions.customize(isStressTest: Boolean): ModelCheckingOptions = this open fun StressOptions.customize(isStressTest: Boolean): StressOptions = this @Test fun modelCheckingTest() = ModelCheckingOptions() - .iterations(if (isStressTest) 100 else 20) + .iterations(if (isStressTest) 200 else 20) .invocationsPerIteration(if (isStressTest) 10_000 else 1_000) .commonConfiguration() .customize(isStressTest) @@ -24,7 +24,7 @@ abstract class AbstractLincheckTest : VerifierState() { @Test fun stressTest() = StressOptions() - .iterations(if (isStressTest) 100 else 20) + .iterations(if (isStressTest) 200 else 20) .invocationsPerIteration(if (isStressTest) 10_000 else 1_000) .commonConfiguration() .customize(isStressTest) @@ -32,10 +32,13 @@ abstract class AbstractLincheckTest : VerifierState() { private fun <O : Options<O, *>> O.commonConfiguration(): O = this .actorsBefore(if (isStressTest) 3 else 1) + // All the bugs we have discovered so far + // were reproducible on at most 3 threads .threads(3) - .actorsPerThread(if (isStressTest) 4 else 2) + // 3 operations per thread is sufficient, + // while increasing this number declines + // the model checking coverage. + .actorsPerThread(if (isStressTest) 3 else 2) .actorsAfter(if (isStressTest) 3 else 0) .customize(isStressTest) - - override fun extractState(): Any = error("Not implemented") } diff --git a/kotlinx-coroutines-core/jvm/test/ConcurrentTestUtilities.kt b/kotlinx-coroutines-core/jvm/test/ConcurrentTestUtilities.kt index b46adda9..4ccb74b4 100644 --- a/kotlinx-coroutines-core/jvm/test/ConcurrentTestUtilities.kt +++ b/kotlinx-coroutines-core/jvm/test/ConcurrentTestUtilities.kt @@ -23,7 +23,7 @@ private object BlackHole { @Suppress("ACTUAL_WITHOUT_EXPECT") internal actual typealias SuppressSupportingThrowable = Throwable -@Suppress("EXTENSION_SHADOWED_BY_MEMBER") +@Suppress("EXTENSION_SHADOWED_BY_MEMBER", "unused") actual fun Throwable.printStackTrace() = printStackTrace() actual fun currentThreadName(): String = Thread.currentThread().name diff --git a/kotlinx-coroutines-core/jvm/test/ExecutorsTest.kt b/kotlinx-coroutines-core/jvm/test/ExecutorsTest.kt index ebf08a03..6e5b18f7 100644 --- a/kotlinx-coroutines-core/jvm/test/ExecutorsTest.kt +++ b/kotlinx-coroutines-core/jvm/test/ExecutorsTest.kt @@ -82,6 +82,22 @@ class ExecutorsTest : TestBase() { } @Test + fun testCustomDispatcherToExecutorDispatchNotNeeded() { + expect(1) + val dispatcher = object : CoroutineDispatcher() { + override fun isDispatchNeeded(context: CoroutineContext) = false + + override fun dispatch(context: CoroutineContext, block: Runnable) { + fail("should not dispatch") + } + } + dispatcher.asExecutor().execute { + expect(2) + } + finish(3) + } + + @Test fun testTwoThreads() { val ctx1 = newSingleThreadContext("Ctx1") val ctx2 = newSingleThreadContext("Ctx2") @@ -106,4 +122,4 @@ class ExecutorsTest : TestBase() { dispatcher.close() check(executorService.isShutdown) } -}
\ No newline at end of file +} diff --git a/kotlinx-coroutines-core/jvm/test/MemoryFootprintTest.kt b/kotlinx-coroutines-core/jvm/test/MemoryFootprintTest.kt new file mode 100644 index 00000000..be467cc5 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/MemoryFootprintTest.kt @@ -0,0 +1,25 @@ +/* + * Copyright 2016-2022 JetBrains s.r.o. Use of this source code is governed by the Apache 2.0 license. + */ + +package kotlinx.coroutines + +import org.junit.Test +import org.openjdk.jol.info.ClassLayout +import kotlin.test.* + + +class MemoryFootprintTest : TestBase(true) { + + @Test + fun testJobLayout() = assertLayout(Job().javaClass, 24) + + @Test + fun testCancellableContinuationFootprint() = assertLayout(CancellableContinuationImpl::class.java, 48) + + private fun assertLayout(clz: Class<*>, expectedSize: Int) { + val size = ClassLayout.parseClass(clz).instanceSize() +// println(ClassLayout.parseClass(clz).toPrintable()) + assertEquals(expectedSize.toLong(), size) + } +} diff --git a/kotlinx-coroutines-core/jvm/test/MutexCancellationStressTest.kt b/kotlinx-coroutines-core/jvm/test/MutexCancellationStressTest.kt new file mode 100644 index 00000000..20798b83 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/MutexCancellationStressTest.kt @@ -0,0 +1,85 @@ +/* + * Copyright 2016-2022 JetBrains s.r.o. Use of this source code is governed by the Apache 2.0 license. + */ + +package kotlinx.coroutines + +import kotlinx.coroutines.internal.* +import kotlinx.coroutines.selects.* +import kotlinx.coroutines.sync.* +import org.junit.* +import org.junit.Test +import java.util.concurrent.* +import java.util.concurrent.atomic.AtomicBoolean +import java.util.concurrent.atomic.AtomicInteger +import kotlin.test.* + +class MutexCancellationStressTest : TestBase() { + @Test + fun testStressCancellationDoesNotBreakMutex() = runTest { + val mutex = Mutex() + val mutexJobNumber = 3 + val mutexOwners = Array(mutexJobNumber) { "$it" } + val dispatcher = Executors.newFixedThreadPool(mutexJobNumber + 2).asCoroutineDispatcher() + var counter = 0 + val counterLocal = Array(mutexJobNumber) { AtomicInteger(0) } + val completed = AtomicBoolean(false) + val mutexJobLauncher: (jobNumber: Int) -> Job = { jobId -> + val coroutineName = "MutexJob-$jobId" + // ATOMIC to always have a chance to proceed + launch(dispatcher + CoroutineName(coroutineName), CoroutineStart.ATOMIC) { + while (!completed.get()) { + // Stress out holdsLock + mutex.holdsLock(mutexOwners[(jobId + 1) % mutexJobNumber]) + // Stress out lock-like primitives + if (mutex.tryLock(mutexOwners[jobId])) { + counterLocal[jobId].incrementAndGet() + counter++ + mutex.unlock(mutexOwners[jobId]) + } + mutex.withLock(mutexOwners[jobId]) { + counterLocal[jobId].incrementAndGet() + counter++ + } + select<Unit> { + mutex.onLock(mutexOwners[jobId]) { + counterLocal[jobId].incrementAndGet() + counter++ + mutex.unlock(mutexOwners[jobId]) + } + } + } + } + } + val mutexJobs = (0 until mutexJobNumber).map { mutexJobLauncher(it) }.toMutableList() + val checkProgressJob = launch(dispatcher + CoroutineName("checkProgressJob")) { + var lastCounterLocalSnapshot = (0 until mutexJobNumber).map { 0 } + while (!completed.get()) { + delay(500) + // If we've caught the completion after delay, then there is a chance no progress were made whatsoever, bail out + if (completed.get()) return@launch + val c = counterLocal.map { it.value } + for (i in 0 until mutexJobNumber) { + assert(c[i] > lastCounterLocalSnapshot[i]) { "No progress in MutexJob-$i, last observed state: ${c[i]}" } + } + lastCounterLocalSnapshot = c + } + } + val cancellationJob = launch(dispatcher + CoroutineName("cancellationJob")) { + var cancellingJobId = 0 + while (!completed.get()) { + val jobToCancel = mutexJobs.removeFirst() + jobToCancel.cancelAndJoin() + mutexJobs += mutexJobLauncher(cancellingJobId) + cancellingJobId = (cancellingJobId + 1) % mutexJobNumber + } + } + delay(2000L * stressTestMultiplier) + completed.set(true) + cancellationJob.join() + mutexJobs.forEach { it.join() } + checkProgressJob.join() + assertEquals(counter, counterLocal.sumOf { it.value }) + dispatcher.close() + } +} diff --git a/kotlinx-coroutines-core/jvm/test/NoParamAssertionsTest.kt b/kotlinx-coroutines-core/jvm/test/NoParamAssertionsTest.kt new file mode 100644 index 00000000..5e1c4625 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/NoParamAssertionsTest.kt @@ -0,0 +1,29 @@ +/* + * Copyright 2016-2022 JetBrains s.r.o. Use of this source code is governed by the Apache 2.0 license. + */ +package kotlinx.coroutines + +import kotlinx.coroutines.* +import org.junit.Test +import kotlin.test.* + + +class NoParamAssertionsTest : TestBase() { + // These tests verify that we haven't omitted "-Xno-param-assertions" and "-Xno-receiver-assertions" + + @Test + fun testNoReceiverAssertion() { + val function: (ThreadLocal<Int>, Int) -> ThreadContextElement<Int> = ThreadLocal<Int>::asContextElement + @Suppress("UNCHECKED_CAST") + val unsafeCasted = function as ((ThreadLocal<Int>?, Int) -> ThreadContextElement<Int>) + unsafeCasted(null, 42) + } + + @Test + fun testNoParamAssertion() { + val function: (ThreadLocal<Any>, Any) -> ThreadContextElement<Any> = ThreadLocal<Any>::asContextElement + @Suppress("UNCHECKED_CAST") + val unsafeCasted = function as ((ThreadLocal<Any?>?, Any?) -> ThreadContextElement<Any>) + unsafeCasted(ThreadLocal.withInitial { Any() }, null) + } +} diff --git a/kotlinx-coroutines-core/jvm/test/ReusableCancellableContinuationInvariantStressTest.kt b/kotlinx-coroutines-core/jvm/test/ReusableCancellableContinuationInvariantStressTest.kt new file mode 100644 index 00000000..4d8116c9 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/ReusableCancellableContinuationInvariantStressTest.kt @@ -0,0 +1,53 @@ +/* + * Copyright 2016-2021 JetBrains s.r.o. Use of this source code is governed by the Apache 2.0 license. + */ + +package kotlinx.coroutines + +import org.junit.Test +import java.util.concurrent.CountDownLatch +import java.util.concurrent.atomic.AtomicReference +import kotlin.coroutines.* + +// Stresses scenario from #3613 +class ReusableCancellableContinuationInvariantStressTest : TestBase() { + + // Tests have a timeout 10 sec because the bug they catch leads to an infinite spin-loop + + @Test(timeout = 10_000) + fun testExceptionFromSuspendReusable() = doTest { /* nothing */ } + + + @Test(timeout = 10_000) + fun testExceptionFromCancelledSuspendReusable() = doTest { it.cancel() } + + + @Suppress("SuspendFunctionOnCoroutineScope") + private inline fun doTest(crossinline block: (Job) -> Unit) { + runTest { + repeat(10_000) { + val latch = CountDownLatch(1) + val continuationToResume = AtomicReference<Continuation<Unit>?>(null) + val j1 = launch(Dispatchers.Default) { + latch.await() + suspendCancellableCoroutineReusable { + continuationToResume.set(it) + block(coroutineContext.job) + throw CancellationException() // Don't let getResult() chance to execute + } + } + + val j2 = launch(Dispatchers.Default) { + latch.await() + while (continuationToResume.get() == null) { + // spin + } + continuationToResume.get()!!.resume(Unit) + } + + latch.countDown() + joinAll(j1, j2) + } + } + } +} diff --git a/kotlinx-coroutines-core/jvm/test/TestBase.kt b/kotlinx-coroutines-core/jvm/test/TestBase.kt index ce94d33a..f9e5466b 100644 --- a/kotlinx-coroutines-core/jvm/test/TestBase.kt +++ b/kotlinx-coroutines-core/jvm/test/TestBase.kt @@ -97,10 +97,10 @@ public actual open class TestBase(private var disableOutCheck: Boolean) { private fun printError(message: String, cause: Throwable) { setError(cause) - println("$message: $cause") - cause.printStackTrace(System.out) - println("--- Detected at ---") - Throwable().printStackTrace(System.out) + System.err.println("$message: $cause") + cause.printStackTrace(System.err) + System.err.println("--- Detected at ---") + Throwable().printStackTrace(System.err) } /** @@ -254,3 +254,10 @@ public actual open class TestBase(private var disableOutCheck: Boolean) { protected suspend fun currentDispatcher() = coroutineContext[ContinuationInterceptor]!! } + +/* + * We ignore tests that test **real** non-virtualized tests with time on Windows, because + * our CI Windows is virtualized itself (oh, the irony) and its clock resolution is dozens of ms, + * which makes such tests flaky. + */ +public actual val isJavaAndWindows: Boolean = System.getProperty("os.name")!!.contains("Windows") diff --git a/kotlinx-coroutines-core/jvm/test/TestBaseExtension.kt b/kotlinx-coroutines-core/jvm/test/TestBaseExtension.kt deleted file mode 100644 index 799e559a..00000000 --- a/kotlinx-coroutines-core/jvm/test/TestBaseExtension.kt +++ /dev/null @@ -1,10 +0,0 @@ -/* - * Copyright 2016-2021 JetBrains s.r.o. Use of this source code is governed by the Apache 2.0 license. - */ -package kotlinx.coroutines - -public actual fun TestBase.runMtTest( - expected: ((Throwable) -> Boolean)?, - unhandled: List<(Throwable) -> Boolean>, - block: suspend CoroutineScope.() -> Unit -): TestResult = runTest(expected, unhandled, block) diff --git a/kotlinx-coroutines-core/jvm/test/ThreadContextElementTest.kt b/kotlinx-coroutines-core/jvm/test/ThreadContextElementTest.kt index ec45406b..3bb79b49 100644 --- a/kotlinx-coroutines-core/jvm/test/ThreadContextElementTest.kt +++ b/kotlinx-coroutines-core/jvm/test/ThreadContextElementTest.kt @@ -7,6 +7,7 @@ package kotlinx.coroutines import org.junit.Test import kotlin.coroutines.* import kotlin.test.* +import kotlinx.coroutines.flow.* class ThreadContextElementTest : TestBase() { @@ -37,7 +38,7 @@ class ThreadContextElementTest : TestBase() { } @Test - fun testUndispatched()= runTest { + fun testUndispatched() = runTest { val exceptionHandler = coroutineContext[CoroutineExceptionHandler]!! val data = MyData() val element = MyElement(data) @@ -156,6 +157,56 @@ class ThreadContextElementTest : TestBase() { } } } + + class JobCaptor(val capturees: ArrayList<Job> = ArrayList()) : ThreadContextElement<Unit> { + + companion object Key : CoroutineContext.Key<MyElement> + + override val key: CoroutineContext.Key<*> get() = Key + + override fun updateThreadContext(context: CoroutineContext) { + capturees.add(context.job) + } + + override fun restoreThreadContext(context: CoroutineContext, oldState: Unit) { + } + } + + @Test + fun testWithContextJobAccess() = runTest { + val captor = JobCaptor() + val manuallyCaptured = ArrayList<Job>() + runBlocking(captor) { + manuallyCaptured += coroutineContext.job + withContext(CoroutineName("undispatched")) { + manuallyCaptured += coroutineContext.job + withContext(Dispatchers.IO) { + manuallyCaptured += coroutineContext.job + } + // Context restored, captured again + manuallyCaptured += coroutineContext.job + } + // Context restored, captured again + manuallyCaptured += coroutineContext.job + } + + assertEquals(manuallyCaptured, captor.capturees) + } + + @Test + fun testThreadLocalFlowOn() = runTest { + val myData = MyData() + myThreadLocal.set(myData) + expect(1) + flow { + assertEquals(myData, myThreadLocal.get()) + emit(1) + } + .flowOn(myThreadLocal.asContextElement() + Dispatchers.Default) + .single() + myThreadLocal.set(null) + finish(2) + } } class MyData @@ -224,6 +275,7 @@ class CopyForChildCoroutineElement(val data: MyData?) : CopyableThreadContextEle } } + /** * Calls [block], setting the value of [this] [ThreadLocal] for the duration of [block]. * diff --git a/kotlinx-coroutines-core/jvm/test/ThreadContextMutableCopiesTest.kt b/kotlinx-coroutines-core/jvm/test/ThreadContextMutableCopiesTest.kt index 34e5955f..73d4ee6e 100644 --- a/kotlinx-coroutines-core/jvm/test/ThreadContextMutableCopiesTest.kt +++ b/kotlinx-coroutines-core/jvm/test/ThreadContextMutableCopiesTest.kt @@ -4,6 +4,7 @@ package kotlinx.coroutines +import kotlinx.coroutines.flow.* import kotlin.coroutines.* import kotlin.test.* @@ -131,4 +132,32 @@ class ThreadContextMutableCopiesTest : TestBase() { finish(2) } } + + @Test + fun testDataIsCopiedThroughFlowOnUndispatched() = runTest { + expect(1) + val root = MyMutableElement(ArrayList()) + val originalData = root.mutableData + flow { + assertNotSame(originalData, threadLocalData.get()) + emit(1) + } + .flowOn(root) + .single() + finish(2) + } + + @Test + fun testDataIsCopiedThroughFlowOnDispatched() = runTest { + expect(1) + val root = MyMutableElement(ArrayList()) + val originalData = root.mutableData + flow { + assertNotSame(originalData, threadLocalData.get()) + emit(1) + } + .flowOn(root + Dispatchers.Default) + .single() + finish(2) + } } diff --git a/kotlinx-coroutines-core/jvm/test/channels/BroadcastChannelLeakTest.kt b/kotlinx-coroutines-core/jvm/test/channels/BroadcastChannelLeakTest.kt index 66b08c74..df944654 100644 --- a/kotlinx-coroutines-core/jvm/test/channels/BroadcastChannelLeakTest.kt +++ b/kotlinx-coroutines-core/jvm/test/channels/BroadcastChannelLeakTest.kt @@ -6,8 +6,8 @@ import kotlin.test.* class BroadcastChannelLeakTest : TestBase() { @Test - fun testArrayBroadcastChannelSubscriptionLeak() { - checkLeak { ArrayBroadcastChannel(1) } + fun testBufferedBroadcastChannelSubscriptionLeak() { + checkLeak { BroadcastChannelImpl(1) } } @Test diff --git a/kotlinx-coroutines-core/jvm/test/channels/ArrayChannelStressTest.kt b/kotlinx-coroutines-core/jvm/test/channels/BufferedChannelStressTest.kt index 74dc24c7..a6464263 100644 --- a/kotlinx-coroutines-core/jvm/test/channels/ArrayChannelStressTest.kt +++ b/kotlinx-coroutines-core/jvm/test/channels/BufferedChannelStressTest.kt @@ -10,7 +10,7 @@ import org.junit.runner.* import org.junit.runners.* @RunWith(Parameterized::class) -class ArrayChannelStressTest(private val capacity: Int) : TestBase() { +class BufferedChannelStressTest(private val capacity: Int) : TestBase() { companion object { @Parameterized.Parameters(name = "{0}, nSenders={1}, nReceivers={2}") diff --git a/kotlinx-coroutines-core/jvm/test/channels/ChannelMemoryLeakStressTest.kt b/kotlinx-coroutines-core/jvm/test/channels/ChannelMemoryLeakStressTest.kt new file mode 100644 index 00000000..ebc2bee8 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/channels/ChannelMemoryLeakStressTest.kt @@ -0,0 +1,25 @@ +/* + * Copyright 2016-2022 JetBrains s.r.o. Use of this source code is governed by the Apache 2.0 license. + */ + +package channels + +import kotlinx.coroutines.* +import kotlinx.coroutines.channels.* +import org.junit.Test + +class ChannelMemoryLeakStressTest : TestBase() { + private val nRepeat = 1_000_000 * stressTestMultiplier + + @Test + fun test() = runTest { + val c = Channel<Any>(1) + repeat(nRepeat) { + c.send(bigValue()) + c.receive() + } + } + + // capture big value for fast OOM in case of a bug + private fun bigValue(): ByteArray = ByteArray(4096) +} diff --git a/kotlinx-coroutines-core/jvm/test/channels/ChannelSendReceiveStressTest.kt b/kotlinx-coroutines-core/jvm/test/channels/ChannelSendReceiveStressTest.kt index 7e55f2e6..8a60ce50 100644 --- a/kotlinx-coroutines-core/jvm/test/channels/ChannelSendReceiveStressTest.kt +++ b/kotlinx-coroutines-core/jvm/test/channels/ChannelSendReceiveStressTest.kt @@ -7,12 +7,14 @@ package kotlinx.coroutines.channels import kotlinx.coroutines.* import kotlinx.coroutines.selects.* import org.junit.* +import org.junit.Ignore import org.junit.Test import org.junit.runner.* import org.junit.runners.* import java.util.concurrent.atomic.* import kotlin.test.* +@Ignore @RunWith(Parameterized::class) class ChannelSendReceiveStressTest( private val kind: TestChannelKind, @@ -25,10 +27,7 @@ class ChannelSendReceiveStressTest( fun params(): Collection<Array<Any>> = listOf(1, 2, 10).flatMap { nSenders -> listOf(1, 10).flatMap { nReceivers -> - TestChannelKind.values() - // Workaround for bug that won't be fixed unless new channel implementation, see #2443 - .filter { it != TestChannelKind.LINKED_LIST } - .map { arrayOf(it, nSenders, nReceivers) } + TestChannelKind.values().map { arrayOf(it, nSenders, nReceivers) } } } } @@ -36,7 +35,7 @@ class ChannelSendReceiveStressTest( private val timeLimit = 30_000L * stressTestMultiplier // 30 sec private val nEvents = 200_000 * stressTestMultiplier - private val maxBuffer = 10_000 // artificial limit for LinkedListChannel + private val maxBuffer = 10_000 // artificial limit for unlimited channel val channel = kind.create<Int>() private val sendersCompleted = AtomicInteger() @@ -107,6 +106,7 @@ class ChannelSendReceiveStressTest( repeat(nReceivers) { receiveIndex -> println(" Received by #$receiveIndex ${receivedBy[receiveIndex]}") } + (channel as? BufferedChannel<*>)?.checkSegmentStructureInvariants() assertEquals(nSenders, sendersCompleted.get()) assertEquals(nReceivers, receiversCompleted.get()) assertEquals(0, dupes.get()) @@ -121,7 +121,7 @@ class ChannelSendReceiveStressTest( sentTotal.incrementAndGet() if (!kind.isConflated) { while (sentTotal.get() > receivedTotal.get() + maxBuffer) - yield() // throttle fast senders to prevent OOM with LinkedListChannel + yield() // throttle fast senders to prevent OOM with an unlimited channel } } diff --git a/kotlinx-coroutines-core/jvm/test/channels/ChannelUndeliveredElementSelectOldStressTest.kt b/kotlinx-coroutines-core/jvm/test/channels/ChannelUndeliveredElementSelectOldStressTest.kt new file mode 100644 index 00000000..25cccf94 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/channels/ChannelUndeliveredElementSelectOldStressTest.kt @@ -0,0 +1,254 @@ +/* + * Copyright 2016-2021 JetBrains s.r.o. Use of this source code is governed by the Apache 2.0 license. + */ + +package kotlinx.coroutines.channels + +import kotlinx.atomicfu.* +import kotlinx.coroutines.* +import kotlinx.coroutines.selects.* +import org.junit.After +import org.junit.Test +import org.junit.runner.* +import org.junit.runners.* +import kotlin.random.Random +import kotlin.test.* + +/** + * Tests resource transfer via channel send & receive operations, including their select versions, + * using `onUndeliveredElement` to detect lost resources and close them properly. + */ +@RunWith(Parameterized::class) +class ChannelUndeliveredElementSelectOldStressTest(private val kind: TestChannelKind) : TestBase() { + companion object { + @Parameterized.Parameters(name = "{0}") + @JvmStatic + fun params(): Collection<Array<Any>> = + TestChannelKind.values() + .filter { !it.viaBroadcast } + .map { arrayOf<Any>(it) } + } + + private val iterationDurationMs = 100L + private val testIterations = 20 * stressTestMultiplier // 2 sec + + private val dispatcher = newFixedThreadPoolContext(2, "ChannelAtomicCancelStressTest") + private val scope = CoroutineScope(dispatcher) + + private val channel = kind.create<Data> { it.failedToDeliver() } + private val senderDone = Channel<Boolean>(1) + private val receiverDone = Channel<Boolean>(1) + + @Volatile + private var lastReceived = -1L + + private var stoppedSender = 0L + private var stoppedReceiver = 0L + + private var sentCnt = 0L // total number of send attempts + private var receivedCnt = 0L // actually received successfully + private var dupCnt = 0L // duplicates (should never happen) + private val failedToDeliverCnt = atomic(0L) // out of sent + + private val modulo = 1 shl 25 + private val mask = (modulo - 1).toLong() + private val sentStatus = ItemStatus() // 1 - send norm, 2 - send select, +2 - did not throw exception + private val receivedStatus = ItemStatus() // 1-6 received + private val failedStatus = ItemStatus() // 1 - failed + + lateinit var sender: Job + lateinit var receiver: Job + + @After + fun tearDown() { + dispatcher.close() + } + + private inline fun cancellable(done: Channel<Boolean>, block: () -> Unit) { + try { + block() + } finally { + if (!done.trySend(true).isSuccess) + error(IllegalStateException("failed to offer to done channel")) + } + } + + @Test + fun testAtomicCancelStress() = runBlocking { + println("=== ChannelAtomicCancelStressTest $kind") + var nextIterationTime = System.currentTimeMillis() + iterationDurationMs + var iteration = 0 + launchSender() + launchReceiver() + while (!hasError()) { + if (System.currentTimeMillis() >= nextIterationTime) { + nextIterationTime += iterationDurationMs + iteration++ + verify(iteration) + if (iteration % 10 == 0) printProgressSummary(iteration) + if (iteration >= testIterations) break + launchSender() + launchReceiver() + } + when (Random.nextInt(3)) { + 0 -> { // cancel & restart sender + stopSender() + launchSender() + } + 1 -> { // cancel & restart receiver + stopReceiver() + launchReceiver() + } + 2 -> yield() // just yield (burn a little time) + } + } + } + + private suspend fun verify(iteration: Int) { + stopSender() + drainReceiver() + stopReceiver() + try { + assertEquals(0, dupCnt) + assertEquals(sentCnt - failedToDeliverCnt.value, receivedCnt) + } catch (e: Throwable) { + printProgressSummary(iteration) + printErrorDetails() + throw e + } + sentStatus.clear() + receivedStatus.clear() + failedStatus.clear() + } + + private fun printProgressSummary(iteration: Int) { + println("--- ChannelAtomicCancelStressTest $kind -- $iteration of $testIterations") + println(" Sent $sentCnt times to channel") + println(" Received $receivedCnt times from channel") + println(" Failed to deliver ${failedToDeliverCnt.value} times") + println(" Stopped sender $stoppedSender times") + println(" Stopped receiver $stoppedReceiver times") + println(" Duplicated $dupCnt deliveries") + } + + private fun printErrorDetails() { + val min = minOf(sentStatus.min, receivedStatus.min, failedStatus.min) + val max = maxOf(sentStatus.max, receivedStatus.max, failedStatus.max) + for (x in min..max) { + val sentCnt = if (sentStatus[x] != 0) 1 else 0 + val receivedCnt = if (receivedStatus[x] != 0) 1 else 0 + val failedToDeliverCnt = failedStatus[x] + if (sentCnt - failedToDeliverCnt != receivedCnt) { + println("!!! Error for value $x: " + + "sentStatus=${sentStatus[x]}, " + + "receivedStatus=${receivedStatus[x]}, " + + "failedStatus=${failedStatus[x]}" + ) + } + } + } + + + private fun launchSender() { + sender = scope.launch(start = CoroutineStart.ATOMIC) { + cancellable(senderDone) { + var counter = 0 + while (true) { + val trySendData = Data(sentCnt++) + sentStatus[trySendData.x] = 1 + selectOld<Unit> { channel.onSend(trySendData) {} } + sentStatus[trySendData.x] = 3 + when { + // must artificially slow down LINKED_LIST sender to avoid overwhelming receiver and going OOM + kind == TestChannelKind.UNLIMITED -> while (sentCnt > lastReceived + 100) yield() + // yield periodically to check cancellation on conflated channels + kind.isConflated -> if (counter++ % 100 == 0) yield() + } + } + } + } + } + + private suspend fun stopSender() { + stoppedSender++ + sender.cancelAndJoin() + senderDone.receive() + } + + private fun launchReceiver() { + receiver = scope.launch(start = CoroutineStart.ATOMIC) { + cancellable(receiverDone) { + while (true) { + selectOld<Unit> { + channel.onReceive { receivedData -> + receivedData.onReceived() + receivedCnt++ + val received = receivedData.x + if (received <= lastReceived) + dupCnt++ + lastReceived = received + receivedStatus[received] = 1 + } + } + } + } + } + } + + private suspend fun drainReceiver() { + while (!channel.isEmpty) yield() // burn time until receiver gets it all + } + + private suspend fun stopReceiver() { + stoppedReceiver++ + receiver.cancelAndJoin() + receiverDone.receive() + } + + private inner class Data(val x: Long) { + private val firstFailedToDeliverOrReceivedCallTrace = atomic<Exception?>(null) + + fun failedToDeliver() { + val trace = if (TRACING_ENABLED) Exception("First onUndeliveredElement() call") else DUMMY_TRACE_EXCEPTION + if (firstFailedToDeliverOrReceivedCallTrace.compareAndSet(null, trace)) { + failedToDeliverCnt.incrementAndGet() + failedStatus[x] = 1 + return + } + throw IllegalStateException("onUndeliveredElement()/onReceived() notified twice", firstFailedToDeliverOrReceivedCallTrace.value!!) + } + + fun onReceived() { + val trace = if (TRACING_ENABLED) Exception("First onReceived() call") else DUMMY_TRACE_EXCEPTION + if (firstFailedToDeliverOrReceivedCallTrace.compareAndSet(null, trace)) return + throw IllegalStateException("onUndeliveredElement()/onReceived() notified twice", firstFailedToDeliverOrReceivedCallTrace.value!!) + } + } + + inner class ItemStatus { + private val a = ByteArray(modulo) + private val _min = atomic(Long.MAX_VALUE) + private val _max = atomic(-1L) + + val min: Long get() = _min.value + val max: Long get() = _max.value + + operator fun set(x: Long, value: Int) { + a[(x and mask).toInt()] = value.toByte() + _min.update { y -> minOf(x, y) } + _max.update { y -> maxOf(x, y) } + } + + operator fun get(x: Long): Int = a[(x and mask).toInt()].toInt() + + fun clear() { + if (_max.value < 0) return + for (x in _min.value.._max.value) a[(x and mask).toInt()] = 0 + _min.value = Long.MAX_VALUE + _max.value = -1L + } + } +} + +private const val TRACING_ENABLED = false // Change to `true` to enable the tracing +private val DUMMY_TRACE_EXCEPTION = Exception("The tracing is disabled; please enable it by changing the `TRACING_ENABLED` constant to `true`.") diff --git a/kotlinx-coroutines-core/jvm/test/channels/ChannelUndeliveredElementStressTest.kt b/kotlinx-coroutines-core/jvm/test/channels/ChannelUndeliveredElementStressTest.kt index 12334326..f8a56447 100644 --- a/kotlinx-coroutines-core/jvm/test/channels/ChannelUndeliveredElementStressTest.kt +++ b/kotlinx-coroutines-core/jvm/test/channels/ChannelUndeliveredElementStressTest.kt @@ -116,6 +116,7 @@ class ChannelUndeliveredElementStressTest(private val kind: TestChannelKind) : T printErrorDetails() throw e } + (channel as? BufferedChannel<*>)?.checkSegmentStructureInvariants() sentStatus.clear() receivedStatus.clear() failedStatus.clear() @@ -165,7 +166,7 @@ class ChannelUndeliveredElementStressTest(private val kind: TestChannelKind) : T sentStatus[trySendData.x] = sendMode + 2 when { // must artificially slow down LINKED_LIST sender to avoid overwhelming receiver and going OOM - kind == TestChannelKind.LINKED_LIST -> while (sentCnt > lastReceived + 100) yield() + kind == TestChannelKind.UNLIMITED -> while (sentCnt > lastReceived + 100) yield() // yield periodically to check cancellation on conflated channels kind.isConflated -> if (counter++ % 100 == 0) yield() } @@ -176,7 +177,7 @@ class ChannelUndeliveredElementStressTest(private val kind: TestChannelKind) : T private suspend fun stopSender() { stoppedSender++ - sender.cancel() + sender.cancelAndJoin() senderDone.receive() } @@ -198,6 +199,7 @@ class ChannelUndeliveredElementStressTest(private val kind: TestChannelKind) : T } else -> error("cannot happen") } + receivedData.onReceived() receivedCnt++ val received = receivedData.x if (received <= lastReceived) @@ -220,12 +222,22 @@ class ChannelUndeliveredElementStressTest(private val kind: TestChannelKind) : T } private inner class Data(val x: Long) { - private val failedToDeliver = atomic(false) + private val firstFailedToDeliverOrReceivedCallTrace = atomic<Exception?>(null) fun failedToDeliver() { - check(failedToDeliver.compareAndSet(false, true)) { "onUndeliveredElement notified twice" } - failedToDeliverCnt.incrementAndGet() - failedStatus[x] = 1 + val trace = if (TRACING_ENABLED) Exception("First onUndeliveredElement() call") else DUMMY_TRACE_EXCEPTION + if (firstFailedToDeliverOrReceivedCallTrace.compareAndSet(null, trace)) { + failedToDeliverCnt.incrementAndGet() + failedStatus[x] = 1 + return + } + throw IllegalStateException("onUndeliveredElement()/onReceived() notified twice", firstFailedToDeliverOrReceivedCallTrace.value!!) + } + + fun onReceived() { + val trace = if (TRACING_ENABLED) Exception("First onReceived() call") else DUMMY_TRACE_EXCEPTION + if (firstFailedToDeliverOrReceivedCallTrace.compareAndSet(null, trace)) return + throw IllegalStateException("onUndeliveredElement()/onReceived() notified twice", firstFailedToDeliverOrReceivedCallTrace.value!!) } } @@ -253,3 +265,6 @@ class ChannelUndeliveredElementStressTest(private val kind: TestChannelKind) : T } } } + +private const val TRACING_ENABLED = false // Change to `true` to enable the tracing +private val DUMMY_TRACE_EXCEPTION = Exception("The tracing is disabled; please enable it by changing the `TRACING_ENABLED` constant to `true`.") diff --git a/kotlinx-coroutines-core/jvm/test/channels/InvokeOnCloseStressTest.kt b/kotlinx-coroutines-core/jvm/test/channels/InvokeOnCloseStressTest.kt index 888522c6..8ac85913 100644 --- a/kotlinx-coroutines-core/jvm/test/channels/InvokeOnCloseStressTest.kt +++ b/kotlinx-coroutines-core/jvm/test/channels/InvokeOnCloseStressTest.kt @@ -5,7 +5,6 @@ package kotlinx.coroutines.channels import kotlinx.coroutines.* -import kotlinx.coroutines.channels.* import org.junit.* import org.junit.Test import java.util.concurrent.* @@ -28,7 +27,7 @@ class InvokeOnCloseStressTest : TestBase(), CoroutineScope { @Test fun testInvokedExactlyOnce() = runBlocking { - runStressTest(TestChannelKind.ARRAY_1) + runStressTest(TestChannelKind.BUFFERED_1) } @Test diff --git a/kotlinx-coroutines-core/jvm/test/examples/example-delay-duration-01.kt b/kotlinx-coroutines-core/jvm/test/examples/example-delay-duration-01.kt index 8b3d2d24..954af066 100644 --- a/kotlinx-coroutines-core/jvm/test/examples/example-delay-duration-01.kt +++ b/kotlinx-coroutines-core/jvm/test/examples/example-delay-duration-01.kt @@ -5,7 +5,6 @@ // This file was automatically generated from Delay.kt by Knit tool. Do not edit. package kotlinx.coroutines.examples.exampleDelayDuration01 -import kotlin.time.* import kotlinx.coroutines.* import kotlinx.coroutines.flow.* import kotlin.time.Duration.Companion.milliseconds diff --git a/kotlinx-coroutines-core/jvm/test/examples/example-delay-duration-02.kt b/kotlinx-coroutines-core/jvm/test/examples/example-delay-duration-02.kt index 6500ecd3..45935a09 100644 --- a/kotlinx-coroutines-core/jvm/test/examples/example-delay-duration-02.kt +++ b/kotlinx-coroutines-core/jvm/test/examples/example-delay-duration-02.kt @@ -5,7 +5,6 @@ // This file was automatically generated from Delay.kt by Knit tool. Do not edit. package kotlinx.coroutines.examples.exampleDelayDuration02 -import kotlin.time.* import kotlinx.coroutines.* import kotlinx.coroutines.flow.* import kotlin.time.Duration.Companion.milliseconds diff --git a/kotlinx-coroutines-core/jvm/test/examples/example-delay-duration-03.kt b/kotlinx-coroutines-core/jvm/test/examples/example-delay-duration-03.kt index 4d5e40d7..fc389c24 100644 --- a/kotlinx-coroutines-core/jvm/test/examples/example-delay-duration-03.kt +++ b/kotlinx-coroutines-core/jvm/test/examples/example-delay-duration-03.kt @@ -5,7 +5,6 @@ // This file was automatically generated from Delay.kt by Knit tool. Do not edit. package kotlinx.coroutines.examples.exampleDelayDuration03 -import kotlin.time.* import kotlinx.coroutines.* import kotlinx.coroutines.flow.* import kotlin.time.Duration.Companion.milliseconds diff --git a/kotlinx-coroutines-core/jvm/test/examples/example-timeout-duration-01.kt b/kotlinx-coroutines-core/jvm/test/examples/example-timeout-duration-01.kt new file mode 100644 index 00000000..5db6e6a5 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/examples/example-timeout-duration-01.kt @@ -0,0 +1,27 @@ +/* + * Copyright 2016-2020 JetBrains s.r.o. Use of this source code is governed by the Apache 2.0 license. + */ + +// This file was automatically generated from Delay.kt by Knit tool. Do not edit. +package kotlinx.coroutines.examples.exampleTimeoutDuration01 + +import kotlinx.coroutines.* +import kotlinx.coroutines.flow.* +import kotlin.time.Duration.Companion.milliseconds + +fun main() = runBlocking { + +flow { + emit(1) + delay(100) + emit(2) + delay(100) + emit(3) + delay(1000) + emit(4) +}.timeout(100.milliseconds).catch { + emit(-1) // Item to emit on timeout +}.onEach { + delay(300) // This will not cause a timeout +} +.toList().joinToString().let { println(it) } } diff --git a/kotlinx-coroutines-core/jvm/test/examples/test/FlowDelayTest.kt b/kotlinx-coroutines-core/jvm/test/examples/test/FlowDelayTest.kt index 99e72eb2..f7e93b34 100644 --- a/kotlinx-coroutines-core/jvm/test/examples/test/FlowDelayTest.kt +++ b/kotlinx-coroutines-core/jvm/test/examples/test/FlowDelayTest.kt @@ -50,4 +50,11 @@ class FlowDelayTest { "1, 3, 5, 7, 9" ) } + + @Test + fun testExampleTimeoutDuration01() { + test("ExampleTimeoutDuration01") { kotlinx.coroutines.examples.exampleTimeoutDuration01.main() }.verifyLines( + "1, 2, 3, -1" + ) + } } diff --git a/kotlinx-coroutines-core/jvm/test/exceptions/StackTraceRecoveryCustomExceptionsTest.kt b/kotlinx-coroutines-core/jvm/test/exceptions/StackTraceRecoveryCustomExceptionsTest.kt index d4e19040..0f987e56 100644 --- a/kotlinx-coroutines-core/jvm/test/exceptions/StackTraceRecoveryCustomExceptionsTest.kt +++ b/kotlinx-coroutines-core/jvm/test/exceptions/StackTraceRecoveryCustomExceptionsTest.kt @@ -88,6 +88,26 @@ class StackTraceRecoveryCustomExceptionsTest : TestBase() { } @Test + fun testNestedExceptionWithCause() = runTest { + val result = runCatching { + coroutineScope<Unit> { + throw NestedException(IllegalStateException("ERROR")) + } + } + val ex = result.exceptionOrNull() ?: error("Expected to fail") + assertIs<NestedException>(ex) + assertIs<NestedException>(ex.cause) + val originalCause = ex.cause?.cause + assertIs<IllegalStateException>(originalCause) + assertEquals("ERROR", originalCause.message) + } + + class NestedException : RuntimeException { + constructor(cause: Throwable) : super(cause) + constructor() : super() + } + + @Test fun testWrongMessageExceptionInChannel() = runTest { val result = produce<Unit>(SupervisorJob() + Dispatchers.Unconfined) { throw WrongMessageException("OK") diff --git a/kotlinx-coroutines-core/jvm/test/exceptions/StackTraceRecoveryNestedScopesTest.kt b/kotlinx-coroutines-core/jvm/test/exceptions/StackTraceRecoveryNestedScopesTest.kt index a85bb7a2..dbb1ead5 100644 --- a/kotlinx-coroutines-core/jvm/test/exceptions/StackTraceRecoveryNestedScopesTest.kt +++ b/kotlinx-coroutines-core/jvm/test/exceptions/StackTraceRecoveryNestedScopesTest.kt @@ -12,7 +12,7 @@ class StackTraceRecoveryNestedScopesTest : TestBase() { "\tat kotlinx.coroutines.exceptions.StackTraceRecoveryNestedScopesTest.failure(StackTraceRecoveryNestedScopesTest.kt:9)\n" + "\tat kotlinx.coroutines.exceptions.StackTraceRecoveryNestedScopesTest.access\$failure(StackTraceRecoveryNestedScopesTest.kt:7)\n" + "\tat kotlinx.coroutines.exceptions.StackTraceRecoveryNestedScopesTest\$createFailingAsync\$1.invokeSuspend(StackTraceRecoveryNestedScopesTest.kt:12)\n" + - "\t(Coroutine boundary)\n" + + "\tat _COROUTINE._BOUNDARY._(CoroutineDebugging.kt)\n" + "\tat kotlinx.coroutines.exceptions.StackTraceRecoveryNestedScopesTest\$callWithTimeout\$2.invokeSuspend(StackTraceRecoveryNestedScopesTest.kt:23)\n" + "\tat kotlinx.coroutines.exceptions.StackTraceRecoveryNestedScopesTest\$callCoroutineScope\$2.invokeSuspend(StackTraceRecoveryNestedScopesTest.kt:29)\n" + "\tat kotlinx.coroutines.exceptions.StackTraceRecoveryNestedScopesTest\$$TEST_MACROS\$1.invokeSuspend(StackTraceRecoveryNestedScopesTest.kt:36)\n" + @@ -82,7 +82,7 @@ class StackTraceRecoveryNestedScopesTest : TestBase() { "\tat kotlinx.coroutines.exceptions.StackTraceRecoveryNestedScopesTest.failure(StackTraceRecoveryNestedScopesTest.kt:23)\n" + "\tat kotlinx.coroutines.exceptions.StackTraceRecoveryNestedScopesTest.access\$failure(StackTraceRecoveryNestedScopesTest.kt:7)\n" + "\tat kotlinx.coroutines.exceptions.StackTraceRecoveryNestedScopesTest\$createFailingAsync\$1.invokeSuspend(StackTraceRecoveryNestedScopesTest.kt:26)\n" + - "\t(Coroutine boundary)\n" + + "\tat _COROUTINE._BOUNDARY._(CoroutineDebugging.kt)\n" + "\tat kotlinx.coroutines.exceptions.StackTraceRecoveryNestedScopesTest\$callWithTimeout\$2.invokeSuspend(StackTraceRecoveryNestedScopesTest.kt:37)\n" + "\tat kotlinx.coroutines.exceptions.StackTraceRecoveryNestedScopesTest\$callCoroutineScope\$2.invokeSuspend(StackTraceRecoveryNestedScopesTest.kt:43)\n" + "\tat kotlinx.coroutines.exceptions.StackTraceRecoveryNestedScopesTest\$testAwaitNestedScopes\$1\$deferred\$1.invokeSuspend(StackTraceRecoveryNestedScopesTest.kt:68)\n" + diff --git a/kotlinx-coroutines-core/jvm/test/exceptions/StackTraceRecoverySelectTest.kt b/kotlinx-coroutines-core/jvm/test/exceptions/StackTraceRecoverySelectTest.kt index 0d7648c5..0efa252e 100644 --- a/kotlinx-coroutines-core/jvm/test/exceptions/StackTraceRecoverySelectTest.kt +++ b/kotlinx-coroutines-core/jvm/test/exceptions/StackTraceRecoverySelectTest.kt @@ -5,6 +5,7 @@ package kotlinx.coroutines.exceptions import kotlinx.coroutines.* +import kotlinx.coroutines.channels.* import kotlinx.coroutines.selects.* import org.junit.* import org.junit.rules.* @@ -27,7 +28,7 @@ class StackTraceRecoverySelectTest : TestBase() { val job = CompletableDeferred(Unit) return select { job.onJoin { - yield() // Hide the stackstrace + yield() // Hide the stacktrace expect(2) throw RecoverableTestException() } @@ -50,4 +51,21 @@ class StackTraceRecoverySelectTest : TestBase() { } } } + + @Test + fun testSelectOnReceive() = runTest { + val c = Channel<Unit>() + c.close() + val result = kotlin.runCatching { doSelectOnReceive(c) } + verifyStackTrace("select/${name.methodName}", result.exceptionOrNull()!!) + } + + private suspend fun doSelectOnReceive(c: Channel<Unit>) { + // The channel is closed, should throw an exception + select<Unit> { + c.onReceive { + expectUnreached() + } + } + } } diff --git a/kotlinx-coroutines-core/jvm/test/exceptions/StackTraceRecoveryTest.kt b/kotlinx-coroutines-core/jvm/test/exceptions/StackTraceRecoveryTest.kt index 0a8b6530..1db7c1db 100644 --- a/kotlinx-coroutines-core/jvm/test/exceptions/StackTraceRecoveryTest.kt +++ b/kotlinx-coroutines-core/jvm/test/exceptions/StackTraceRecoveryTest.kt @@ -35,7 +35,7 @@ class StackTraceRecoveryTest : TestBase() { val traces = listOf( "java.util.concurrent.ExecutionException\n" + "\tat kotlinx.coroutines.exceptions.StackTraceRecoveryTest\$testAsync\$1\$createDeferred\$1.invokeSuspend(StackTraceRecoveryTest.kt:99)\n" + - "\t(Coroutine boundary)\n" + + "\tat _COROUTINE._BOUNDARY._(CoroutineDebugging.kt)\n" + "\tat kotlinx.coroutines.exceptions.StackTraceRecoveryTest.oneMoreNestedMethod(StackTraceRecoveryTest.kt:49)\n" + "\tat kotlinx.coroutines.exceptions.StackTraceRecoveryTest.nestedMethod(StackTraceRecoveryTest.kt:44)\n" + "\tat kotlinx.coroutines.exceptions.StackTraceRecoveryTest\$testAsync\$1.invokeSuspend(StackTraceRecoveryTest.kt:17)\n", @@ -57,7 +57,7 @@ class StackTraceRecoveryTest : TestBase() { val stacktrace = listOf( "java.util.concurrent.ExecutionException\n" + "\tat kotlinx.coroutines.exceptions.StackTraceRecoveryTest\$testCompletedAsync\$1\$deferred\$1.invokeSuspend(StackTraceRecoveryTest.kt:44)\n" + - "\t(Coroutine boundary)\n" + + "\tat _COROUTINE._BOUNDARY._(CoroutineDebugging.kt)\n" + "\tat kotlinx.coroutines.exceptions.StackTraceRecoveryTest.oneMoreNestedMethod(StackTraceRecoveryTest.kt:81)\n" + "\tat kotlinx.coroutines.exceptions.StackTraceRecoveryTest.nestedMethod(StackTraceRecoveryTest.kt:75)\n" + "\tat kotlinx.coroutines.exceptions.StackTraceRecoveryTest\$testCompletedAsync\$1.invokeSuspend(StackTraceRecoveryTest.kt:71)", @@ -91,7 +91,7 @@ class StackTraceRecoveryTest : TestBase() { outerMethod(deferred, "kotlinx.coroutines.RecoverableTestException\n" + "\tat kotlinx.coroutines.exceptions.StackTraceRecoveryTest\$testWithContext\$1\$deferred\$1.invokeSuspend(StackTraceRecoveryTest.kt:143)\n" + - "\t(Coroutine boundary)\n" + + "\tat _COROUTINE._BOUNDARY._(CoroutineDebugging.kt)\n" + "\tat kotlinx.coroutines.exceptions.StackTraceRecoveryTest.innerMethod(StackTraceRecoveryTest.kt:158)\n" + "\tat kotlinx.coroutines.exceptions.StackTraceRecoveryTest\$outerMethod\$2.invokeSuspend(StackTraceRecoveryTest.kt:151)\n" + "\tat kotlinx.coroutines.exceptions.StackTraceRecoveryTest.outerMethod(StackTraceRecoveryTest.kt:150)\n" + @@ -128,7 +128,7 @@ class StackTraceRecoveryTest : TestBase() { outerScopedMethod(deferred, "kotlinx.coroutines.RecoverableTestException\n" + "\tat kotlinx.coroutines.exceptions.StackTraceRecoveryTest\$testCoroutineScope\$1\$deferred\$1.invokeSuspend(StackTraceRecoveryTest.kt:143)\n" + - "\t(Coroutine boundary)\n" + + "\tat _COROUTINE._BOUNDARY._(CoroutineDebugging.kt)\n" + "\tat kotlinx.coroutines.exceptions.StackTraceRecoveryTest.innerMethod(StackTraceRecoveryTest.kt:158)\n" + "\tat kotlinx.coroutines.exceptions.StackTraceRecoveryTest\$outerScopedMethod\$2\$1.invokeSuspend(StackTraceRecoveryTest.kt:193)\n" + "\tat kotlinx.coroutines.exceptions.StackTraceRecoveryTest\$outerScopedMethod\$2.invokeSuspend(StackTraceRecoveryTest.kt:151)\n" + @@ -227,7 +227,7 @@ class StackTraceRecoveryTest : TestBase() { "\tat kotlinx.coroutines.exceptions.StackTraceRecoveryTest.throws(StackTraceRecoveryTest.kt:280)\n" + "\tat kotlinx.coroutines.exceptions.StackTraceRecoveryTest.access\$throws(StackTraceRecoveryTest.kt:20)\n" + "\tat kotlinx.coroutines.exceptions.StackTraceRecoveryTest\$throws\$1.invokeSuspend(StackTraceRecoveryTest.kt)\n" + - "\t(Coroutine boundary)\n" + + "\tat _COROUTINE._BOUNDARY._(CoroutineDebugging.kt)\n" + "\tat kotlinx.coroutines.exceptions.StackTraceRecoveryTest.awaiter(StackTraceRecoveryTest.kt:285)\n" + "\tat kotlinx.coroutines.exceptions.StackTraceRecoveryTest\$testNonDispatchedRecovery\$await\$1.invokeSuspend(StackTraceRecoveryTest.kt:291)\n" + "Caused by: kotlinx.coroutines.RecoverableTestException") @@ -244,7 +244,7 @@ class StackTraceRecoveryTest : TestBase() { } catch (e: Throwable) { verifyStackTrace(e, "kotlinx.coroutines.RecoverableTestException\n" + "\tat kotlinx.coroutines.exceptions.StackTraceRecoveryTest\$testCancellableContinuation\$1.invokeSuspend(StackTraceRecoveryTest.kt:329)\n" + - "\t(Coroutine boundary)\n" + + "\tat _COROUTINE._BOUNDARY._(CoroutineDebugging.kt)\n" + "\tat kotlinx.coroutines.exceptions.StackTraceRecoveryTest.awaitCallback(StackTraceRecoveryTest.kt:348)\n" + "\tat kotlinx.coroutines.exceptions.StackTraceRecoveryTest\$testCancellableContinuation\$1\$1.invokeSuspend(StackTraceRecoveryTest.kt:322)\n" + "Caused by: kotlinx.coroutines.RecoverableTestException\n" + diff --git a/kotlinx-coroutines-core/jvm/test/exceptions/Stacktraces.kt b/kotlinx-coroutines-core/jvm/test/exceptions/Stacktraces.kt index f79ad4ba..5d85c9c9 100644 --- a/kotlinx-coroutines-core/jvm/test/exceptions/Stacktraces.kt +++ b/kotlinx-coroutines-core/jvm/test/exceptions/Stacktraces.kt @@ -33,25 +33,10 @@ public fun toStackTrace(t: Throwable): String { } public fun String.normalizeStackTrace(): String = - applyBackspace() - .replace(Regex(":[0-9]+"), "") // remove line numbers + replace(Regex(":[0-9]+"), "") // remove line numbers .replace("kotlinx_coroutines_core_main", "") // yay source sets .replace("kotlinx_coroutines_core", "") .replace(Regex("@[0-9a-f]+"), "") // remove hex addresses in debug toStrings .lines().joinToString("\n") // normalize line separators -public fun String.applyBackspace(): String { - val array = toCharArray() - val stack = CharArray(array.size) - var stackSize = -1 - for (c in array) { - if (c != '\b') { - stack[++stackSize] = c - } else { - --stackSize - } - } - return String(stack, 0, stackSize) -} - public fun String.count(substring: String): Int = split(substring).size - 1
\ No newline at end of file diff --git a/kotlinx-coroutines-core/jvm/test/flow/ConsumeAsFlowLeakTest.kt b/kotlinx-coroutines-core/jvm/test/flow/ConsumeAsFlowLeakTest.kt deleted file mode 100644 index c037be1e..00000000 --- a/kotlinx-coroutines-core/jvm/test/flow/ConsumeAsFlowLeakTest.kt +++ /dev/null @@ -1,48 +0,0 @@ -/* - * Copyright 2016-2019 JetBrains s.r.o. Use of this source code is governed by the Apache 2.0 license. - */ - -package kotlinx.coroutines.flow - -import kotlinx.coroutines.* -import kotlinx.coroutines.channels.* -import org.junit.Test -import kotlin.test.* - -class ConsumeAsFlowLeakTest : TestBase() { - - private data class Box(val i: Int) - - // In companion to avoid references through runTest - companion object { - private val first = Box(4) - private val second = Box(5) - } - - // @Test //ignored until KT-33986 - fun testReferenceIsNotRetained() = testReferenceNotRetained(true) - - @Test - fun testReferenceIsNotRetainedNoSuspension() = testReferenceNotRetained(false) - - private fun testReferenceNotRetained(shouldSuspendOnSend: Boolean) = runTest { - val channel = BroadcastChannel<Box>(1) - val job = launch { - expect(2) - channel.asFlow().collect { - expect(it.i) - } - } - - expect(1) - yield() - expect(3) - channel.send(first) - if (shouldSuspendOnSend) yield() - channel.send(second) - yield() - FieldWalker.assertReachableCount(0, channel) { it === second } - finish(6) - job.cancelAndJoin() - } -} diff --git a/kotlinx-coroutines-core/jvm/test/flow/SharingReferenceTest.kt b/kotlinx-coroutines-core/jvm/test/flow/SharingReferenceTest.kt index 98240fc9..aba95edd 100644 --- a/kotlinx-coroutines-core/jvm/test/flow/SharingReferenceTest.kt +++ b/kotlinx-coroutines-core/jvm/test/flow/SharingReferenceTest.kt @@ -50,7 +50,7 @@ class SharingReferenceTest : TestBase() { @Test fun testStateInSuspendingReference() = runTest { - val flow = weakEmitter.stateIn(GlobalScope) + val flow = weakEmitter.stateIn(ContextScope(executor)) linearize() FieldWalker.assertReachableCount(1, flow) { it === token } } diff --git a/kotlinx-coroutines-core/jvm/test/guide/example-basic-06.kt b/kotlinx-coroutines-core/jvm/test/guide/example-basic-06.kt index 24b890a0..3c503ac1 100644 --- a/kotlinx-coroutines-core/jvm/test/guide/example-basic-06.kt +++ b/kotlinx-coroutines-core/jvm/test/guide/example-basic-06.kt @@ -8,7 +8,7 @@ package kotlinx.coroutines.guide.exampleBasic06 import kotlinx.coroutines.* fun main() = runBlocking { - repeat(100_000) { // launch a lot of coroutines + repeat(50_000) { // launch a lot of coroutines launch { delay(5000L) print(".") diff --git a/kotlinx-coroutines-core/jvm/test/guide/example-cancel-09.kt b/kotlinx-coroutines-core/jvm/test/guide/example-cancel-09.kt index 13910a6c..ecd5a942 100644 --- a/kotlinx-coroutines-core/jvm/test/guide/example-cancel-09.kt +++ b/kotlinx-coroutines-core/jvm/test/guide/example-cancel-09.kt @@ -16,7 +16,7 @@ class Resource { fun main() { runBlocking { - repeat(100_000) { // Launch 100K coroutines + repeat(10_000) { // Launch 10K coroutines launch { val resource = withTimeout(60) { // Timeout of 60 ms delay(50) // Delay for 50 ms diff --git a/kotlinx-coroutines-core/jvm/test/guide/example-cancel-10.kt b/kotlinx-coroutines-core/jvm/test/guide/example-cancel-10.kt index 336f5e3a..03dd8528 100644 --- a/kotlinx-coroutines-core/jvm/test/guide/example-cancel-10.kt +++ b/kotlinx-coroutines-core/jvm/test/guide/example-cancel-10.kt @@ -16,7 +16,7 @@ class Resource { fun main() { runBlocking { - repeat(100_000) { // Launch 100K coroutines + repeat(10_000) { // Launch 10K coroutines launch { var resource: Resource? = null // Not acquired yet try { diff --git a/kotlinx-coroutines-core/jvm/test/guide/example-flow-23.kt b/kotlinx-coroutines-core/jvm/test/guide/example-flow-23.kt index 513fa304..973f3638 100644 --- a/kotlinx-coroutines-core/jvm/test/guide/example-flow-23.kt +++ b/kotlinx-coroutines-core/jvm/test/guide/example-flow-23.kt @@ -16,7 +16,7 @@ fun requestFlow(i: Int): Flow<String> = flow { fun main() = runBlocking<Unit> { val startTime = currentTimeMillis() // remember the start time - (1..3).asFlow().onEach { delay(100) } // a number every 100 ms + (1..3).asFlow().onEach { delay(100) } // emit a number every 100 ms .flatMapConcat { requestFlow(it) } .collect { value -> // collect and print println("$value at ${currentTimeMillis() - startTime} ms from start") diff --git a/kotlinx-coroutines-core/jvm/test/guide/example-select-01.kt b/kotlinx-coroutines-core/jvm/test/guide/example-select-01.kt index c1a962e6..bf15b655 100644 --- a/kotlinx-coroutines-core/jvm/test/guide/example-select-01.kt +++ b/kotlinx-coroutines-core/jvm/test/guide/example-select-01.kt @@ -10,15 +10,15 @@ import kotlinx.coroutines.channels.* import kotlinx.coroutines.selects.* fun CoroutineScope.fizz() = produce<String> { - while (true) { // sends "Fizz" every 300 ms - delay(300) + while (true) { // sends "Fizz" every 500 ms + delay(500) send("Fizz") } } fun CoroutineScope.buzz() = produce<String> { - while (true) { // sends "Buzz!" every 500 ms - delay(500) + while (true) { // sends "Buzz!" every 1000 ms + delay(1000) send("Buzz!") } } diff --git a/kotlinx-coroutines-core/jvm/test/guide/test/BasicsGuideTest.kt b/kotlinx-coroutines-core/jvm/test/guide/test/BasicsGuideTest.kt index 7e54fb1d..550f8e7b 100644 --- a/kotlinx-coroutines-core/jvm/test/guide/test/BasicsGuideTest.kt +++ b/kotlinx-coroutines-core/jvm/test/guide/test/BasicsGuideTest.kt @@ -55,7 +55,7 @@ class BasicsGuideTest { @Test fun testExampleBasic06() { test("ExampleBasic06") { kotlinx.coroutines.guide.exampleBasic06.main() }.also { lines -> - check(lines.size == 1 && lines[0] == ".".repeat(100_000)) + check(lines.size == 1 && lines[0] == ".".repeat(50_000)) } } } diff --git a/kotlinx-coroutines-core/jvm/test/guide/test/SelectGuideTest.kt b/kotlinx-coroutines-core/jvm/test/guide/test/SelectGuideTest.kt index 55650d4c..8bc81913 100644 --- a/kotlinx-coroutines-core/jvm/test/guide/test/SelectGuideTest.kt +++ b/kotlinx-coroutines-core/jvm/test/guide/test/SelectGuideTest.kt @@ -18,7 +18,7 @@ class SelectGuideTest { "fizz -> 'Fizz'", "buzz -> 'Buzz!'", "fizz -> 'Fizz'", - "buzz -> 'Buzz!'" + "fizz -> 'Fizz'" ) } diff --git a/kotlinx-coroutines-core/jvm/test/guide/test/SharedStateGuideTest.kt b/kotlinx-coroutines-core/jvm/test/guide/test/SharedStateGuideTest.kt index 3162b24c..034b82ef 100644 --- a/kotlinx-coroutines-core/jvm/test/guide/test/SharedStateGuideTest.kt +++ b/kotlinx-coroutines-core/jvm/test/guide/test/SharedStateGuideTest.kt @@ -56,12 +56,4 @@ class SharedStateGuideTest { "Counter = 100000" ) } - - @Test - fun testExampleSync07() { - test("ExampleSync07") { kotlinx.coroutines.guide.exampleSync07.main() }.verifyLinesArbitraryTime( - "Completed 100000 actions in xxx ms", - "Counter = 100000" - ) - } } diff --git a/kotlinx-coroutines-core/jvm/test/internal/LockFreeLinkedListAddRemoveStressTest.kt b/kotlinx-coroutines-core/jvm/test/internal/LockFreeLinkedListAddRemoveStressTest.kt deleted file mode 100644 index 3229e664..00000000 --- a/kotlinx-coroutines-core/jvm/test/internal/LockFreeLinkedListAddRemoveStressTest.kt +++ /dev/null @@ -1,56 +0,0 @@ -/* - * Copyright 2016-2020 JetBrains s.r.o. Use of this source code is governed by the Apache 2.0 license. - */ - -package kotlinx.coroutines.internal - -import kotlinx.atomicfu.* -import kotlinx.coroutines.* -import java.util.concurrent.* -import kotlin.concurrent.* -import kotlin.test.* - -class LockFreeLinkedListAddRemoveStressTest : TestBase() { - private class Node : LockFreeLinkedListNode() - - private val nRepeat = 100_000 * stressTestMultiplier - private val list = LockFreeLinkedListHead() - private val barrier = CyclicBarrier(3) - private val done = atomic(false) - private val removed = atomic(0) - - @Test - fun testStressAddRemove() { - val threads = ArrayList<Thread>() - threads += testThread("adder") { - val node = Node() - list.addLast(node) - if (node.remove()) removed.incrementAndGet() - } - threads += testThread("remover") { - val node = list.removeFirstOrNull() - if (node != null) removed.incrementAndGet() - } - try { - for (i in 1..nRepeat) { - barrier.await() - barrier.await() - assertEquals(i, removed.value) - list.validate() - } - } finally { - done.value = true - barrier.await() - threads.forEach { it.join() } - } - } - - private fun testThread(name: String, op: () -> Unit) = thread(name = name) { - while (true) { - barrier.await() - if (done.value) break - op() - barrier.await() - } - } -}
\ No newline at end of file diff --git a/kotlinx-coroutines-core/jvm/test/internal/LockFreeLinkedListLongStressTest.kt b/kotlinx-coroutines-core/jvm/test/internal/LockFreeLinkedListLongStressTest.kt index a70a32b5..7e1b0c6d 100644 --- a/kotlinx-coroutines-core/jvm/test/internal/LockFreeLinkedListLongStressTest.kt +++ b/kotlinx-coroutines-core/jvm/test/internal/LockFreeLinkedListLongStressTest.kt @@ -70,4 +70,4 @@ class LockFreeLinkedListLongStressTest : TestBase() { } require(!expected.hasNext()) } -}
\ No newline at end of file +} diff --git a/kotlinx-coroutines-core/jvm/test/internal/LockFreeLinkedListShortStressTest.kt b/kotlinx-coroutines-core/jvm/test/internal/LockFreeLinkedListShortStressTest.kt deleted file mode 100644 index 2ac51b9b..00000000 --- a/kotlinx-coroutines-core/jvm/test/internal/LockFreeLinkedListShortStressTest.kt +++ /dev/null @@ -1,89 +0,0 @@ -/* - * Copyright 2016-2018 JetBrains s.r.o. Use of this source code is governed by the Apache 2.0 license. - */ - -package kotlinx.coroutines.internal - -import kotlinx.coroutines.* -import org.junit.Test -import java.util.* -import java.util.concurrent.atomic.* -import kotlin.concurrent.* -import kotlin.test.* - -/** - * This stress test has 6 threads adding randomly first to the list and them immediately undoing - * this addition by remove, and 4 threads removing first node. The resulting list that is being - * stressed is very short. - */ -class LockFreeLinkedListShortStressTest : TestBase() { - data class IntNode(val i: Int) : LockFreeLinkedListNode() - val list = LockFreeLinkedListHead() - - private val TEST_DURATION = 5000L * stressTestMultiplier - - val threads = mutableListOf<Thread>() - private val nAdderThreads = 6 - private val nRemoverThreads = 4 - private val completedAdder = AtomicInteger() - private val completedRemover = AtomicInteger() - - private val undone = AtomicInteger() - private val missed = AtomicInteger() - private val removed = AtomicInteger() - - @Test - fun testStress() { - println("--- LockFreeLinkedListShortStressTest") - val deadline = System.currentTimeMillis() + TEST_DURATION - repeat(nAdderThreads) { threadId -> - threads += thread(start = false, name = "adder-$threadId") { - val rnd = Random() - while (System.currentTimeMillis() < deadline) { - var node: IntNode? = IntNode(threadId) - when (rnd.nextInt(3)) { - 0 -> list.addLast(node!!) - 1 -> assertTrue(list.addLastIf(node!!, { true })) // just to test conditional add - 2 -> { // just to test failed conditional add - assertFalse(list.addLastIf(node!!, { false })) - node = null - } - } - if (node != null) { - if (node.remove()) { - undone.incrementAndGet() - } else { - // randomly help other removal's completion - if (rnd.nextBoolean()) node.helpRemove() - missed.incrementAndGet() - } - } - } - completedAdder.incrementAndGet() - } - } - repeat(nRemoverThreads) { threadId -> - threads += thread(start = false, name = "remover-$threadId") { - while (System.currentTimeMillis() < deadline) { - val node = list.removeFirstOrNull() - if (node != null) removed.incrementAndGet() - - } - completedRemover.incrementAndGet() - } - } - threads.forEach { it.start() } - threads.forEach { it.join() } - println("Completed successfully ${completedAdder.get()} adder threads") - println("Completed successfully ${completedRemover.get()} remover threads") - println(" Adders undone ${undone.get()} node additions") - println(" Adders missed ${missed.get()} nodes") - println("Remover removed ${removed.get()} nodes") - assertEquals(nAdderThreads, completedAdder.get()) - assertEquals(nRemoverThreads, completedRemover.get()) - assertEquals(missed.get(), removed.get()) - assertTrue(undone.get() > 0) - assertTrue(missed.get() > 0) - list.validate() - } -}
\ No newline at end of file diff --git a/kotlinx-coroutines-core/jvm/test/internal/OnDemandAllocatingPoolLincheckTest.kt b/kotlinx-coroutines-core/jvm/test/internal/OnDemandAllocatingPoolLincheckTest.kt new file mode 100644 index 00000000..9655890c --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/internal/OnDemandAllocatingPoolLincheckTest.kt @@ -0,0 +1,58 @@ +/* + * Copyright 2016-2022 JetBrains s.r.o. Use of this source code is governed by the Apache 2.0 license. + */ + +package kotlinx.coroutines.internal + +import kotlinx.atomicfu.* +import kotlinx.coroutines.* +import org.jetbrains.kotlinx.lincheck.* +import org.jetbrains.kotlinx.lincheck.annotations.* + +/** + * Test that: + * * All elements allocated in [OnDemandAllocatingPool] get returned when [close] is invoked. + * * After reaching the maximum capacity, new elements are not added. + * * After [close] is invoked, [OnDemandAllocatingPool.allocate] returns `false`. + * * [OnDemandAllocatingPool.close] will return an empty list after the first invocation. + */ +abstract class OnDemandAllocatingPoolLincheckTest(maxCapacity: Int) : AbstractLincheckTest() { + private val counter = atomic(0) + private val pool = OnDemandAllocatingPool(maxCapacity = maxCapacity, create = { + counter.getAndIncrement() + }) + + @Operation + fun allocate(): Boolean = pool.allocate() + + @Operation + fun close(): String = pool.close().sorted().toString() +} + +abstract class OnDemandAllocatingSequentialPool(private val maxCapacity: Int) { + var closed = false + var elements = 0 + + fun allocate() = if (closed) { + false + } else { + if (elements < maxCapacity) { + elements++ + } + true + } + + fun close(): String = if (closed) { + emptyList() + } else { + closed = true + (0 until elements) + }.sorted().toString() +} + +class OnDemandAllocatingPool3LincheckTest : OnDemandAllocatingPoolLincheckTest(3) { + override fun <O : Options<O, *>> O.customize(isStressTest: Boolean): O = + this.sequentialSpecification(OnDemandAllocatingSequentialPool3::class.java) +} + +class OnDemandAllocatingSequentialPool3 : OnDemandAllocatingSequentialPool(3) diff --git a/kotlinx-coroutines-core/jvm/test/internal/SegmentBasedQueue.kt b/kotlinx-coroutines-core/jvm/test/internal/SegmentBasedQueue.kt deleted file mode 100644 index a125bec2..00000000 --- a/kotlinx-coroutines-core/jvm/test/internal/SegmentBasedQueue.kt +++ /dev/null @@ -1,125 +0,0 @@ -/* - * Copyright 2016-2020 JetBrains s.r.o. Use of this source code is governed by the Apache 2.0 license. - */ - -package kotlinx.coroutines.internal - -import kotlinx.atomicfu.* - -/** - * This queue implementation is based on [SegmentList] for testing purposes and is organized as follows. Essentially, - * the [SegmentBasedQueue] is represented as an infinite array of segments, each stores one element (see [OneElementSegment]). - * Both [enqueue] and [dequeue] operations increment the corresponding global index ([enqIdx] for [enqueue] and - * [deqIdx] for [dequeue]) and work with the indexed by this counter cell. Since both operations increment the indices - * at first, there could be a race: [enqueue] increments [enqIdx], then [dequeue] checks that the queue is not empty - * (that's true) and increments [deqIdx], looking into the corresponding cell after that; however, the cell is empty - * because the [enqIdx] operation has not been put its element yet. To make the queue non-blocking, [dequeue] can mark - * the cell with [BROKEN] token and retry the operation, [enqueue] at the same time should restart as well; this way, - * the queue is obstruction-free. - */ -internal class SegmentBasedQueue<T> { - private val head: AtomicRef<OneElementSegment<T>> - private val tail: AtomicRef<OneElementSegment<T>> - - private val enqIdx = atomic(0L) - private val deqIdx = atomic(0L) - - init { - val s = OneElementSegment<T>(0, null, 2) - head = atomic(s) - tail = atomic(s) - } - - // Returns the segments associated with the enqueued element, or `null` if the queue is closed. - fun enqueue(element: T): OneElementSegment<T>? { - while (true) { - val curTail = this.tail.value - val enqIdx = this.enqIdx.getAndIncrement() - val segmentOrClosed = this.tail.findSegmentAndMoveForward(id = enqIdx, startFrom = curTail, createNewSegment = ::createSegment) - if (segmentOrClosed.isClosed) return null - val s = segmentOrClosed.segment - if (s.element.value === BROKEN) continue - if (s.element.compareAndSet(null, element)) return s - } - } - - fun dequeue(): T? { - while (true) { - if (this.deqIdx.value >= this.enqIdx.value) return null - val curHead = this.head.value - val deqIdx = this.deqIdx.getAndIncrement() - val segmentOrClosed = this.head.findSegmentAndMoveForward(id = deqIdx, startFrom = curHead, createNewSegment = ::createSegment) - if (segmentOrClosed.isClosed) return null - val s = segmentOrClosed.segment - if (s.id > deqIdx) continue - var el = s.element.value - if (el === null) { - if (s.element.compareAndSet(null, BROKEN)) continue - else el = s.element.value - } - // The link to the previous segment should be cleaned after retrieving the element; - // otherwise, `close()` cannot clean the slot. - s.cleanPrev() - if (el === BROKEN) continue - @Suppress("UNCHECKED_CAST") - return el as T - } - } - - // `enqueue` should return `null` after the queue is closed - fun close(): OneElementSegment<T> { - val s = this.tail.value.close() - var cur = s - while (true) { - cur.element.compareAndSet(null, BROKEN) - cur = cur.prev ?: break - } - return s - } - - val numberOfSegments: Int get() { - var cur = head.value - var i = 1 - while (true) { - cur = cur.next ?: return i - i++ - } - } - - fun checkHeadPrevIsCleaned() { - check(head.value.prev === null) { "head.prev is not null"} - } - - fun checkAllSegmentsAreNotLogicallyRemoved() { - var prev: OneElementSegment<T>? = null - var cur = head.value - while (true) { - check(!cur.logicallyRemoved || cur.isTail) { - "This queue contains removed segments, memory leak detected" - } - check(cur.prev === prev) { - "Two neighbour segments are incorrectly linked: S.next.prev != S" - } - prev = cur - cur = cur.next ?: return - } - } - -} - -private fun <T> createSegment(id: Long, prev: OneElementSegment<T>?) = OneElementSegment(id, prev, 0) - -internal class OneElementSegment<T>(id: Long, prev: OneElementSegment<T>?, pointers: Int) : Segment<OneElementSegment<T>>(id, prev, pointers) { - val element = atomic<Any?>(null) - - override val maxSlots get() = 1 - - val logicallyRemoved get() = element.value === BROKEN - - fun removeSegment() { - val old = element.getAndSet(BROKEN) - if (old !== BROKEN) onSlotCleaned() - } -} - -private val BROKEN = Symbol("BROKEN")
\ No newline at end of file diff --git a/kotlinx-coroutines-core/jvm/test/internal/SegmentListTest.kt b/kotlinx-coroutines-core/jvm/test/internal/SegmentListTest.kt deleted file mode 100644 index ff6a346c..00000000 --- a/kotlinx-coroutines-core/jvm/test/internal/SegmentListTest.kt +++ /dev/null @@ -1,41 +0,0 @@ -package kotlinx.coroutines.internal - -import kotlinx.atomicfu.* -import org.junit.Test -import kotlin.test.* - -class SegmentListTest { - @Test - fun testRemoveTail() { - val initialSegment = TestSegment(0, null, 2) - val head = AtomicRefHolder(initialSegment) - val tail = AtomicRefHolder(initialSegment) - val s1 = tail.ref.findSegmentAndMoveForward(1, tail.ref.value, ::createTestSegment).segment - assertFalse(s1.removed) - tail.ref.value.onSlotCleaned() - assertFalse(s1.removed) - head.ref.findSegmentAndMoveForward(2, head.ref.value, ::createTestSegment) - assertFalse(s1.removed) - tail.ref.findSegmentAndMoveForward(2, head.ref.value, ::createTestSegment) - assertTrue(s1.removed) - } - - @Test - fun testClose() { - val initialSegment = TestSegment(0, null, 2) - val head = AtomicRefHolder(initialSegment) - val tail = AtomicRefHolder(initialSegment) - tail.ref.findSegmentAndMoveForward(1, tail.ref.value, ::createTestSegment) - assertEquals(tail.ref.value, tail.ref.value.close()) - assertTrue(head.ref.findSegmentAndMoveForward(2, head.ref.value, ::createTestSegment).isClosed) - } -} - -private class AtomicRefHolder<T>(initialValue: T) { - val ref = atomic(initialValue) -} - -private class TestSegment(id: Long, prev: TestSegment?, pointers: Int) : Segment<TestSegment>(id, prev, pointers) { - override val maxSlots: Int get() = 1 -} -private fun createTestSegment(id: Long, prev: TestSegment?) = TestSegment(id, prev, 0)
\ No newline at end of file diff --git a/kotlinx-coroutines-core/jvm/test/internal/SegmentQueueTest.kt b/kotlinx-coroutines-core/jvm/test/internal/SegmentQueueTest.kt deleted file mode 100644 index fd2d3290..00000000 --- a/kotlinx-coroutines-core/jvm/test/internal/SegmentQueueTest.kt +++ /dev/null @@ -1,109 +0,0 @@ -package kotlinx.coroutines.internal - -import kotlinx.coroutines.* -import org.junit.Test -import java.util.* -import java.util.concurrent.CyclicBarrier -import java.util.concurrent.atomic.AtomicInteger -import kotlin.concurrent.thread -import kotlin.random.Random -import kotlin.test.* - -class SegmentQueueTest : TestBase() { - @Test - fun testSimpleTest() { - val q = SegmentBasedQueue<Int>() - assertEquals(1, q.numberOfSegments) - assertNull(q.dequeue()) - q.enqueue(1) - assertEquals(1, q.numberOfSegments) - q.enqueue(2) - assertEquals(2, q.numberOfSegments) - assertEquals(1, q.dequeue()) - assertEquals(2, q.numberOfSegments) - assertEquals(2, q.dequeue()) - assertEquals(1, q.numberOfSegments) - assertNull(q.dequeue()) - } - - @Test - fun testSegmentRemoving() { - val q = SegmentBasedQueue<Int>() - q.enqueue(1) - val s = q.enqueue(2) - q.enqueue(3) - assertEquals(3, q.numberOfSegments) - s!!.removeSegment() - assertEquals(2, q.numberOfSegments) - assertEquals(1, q.dequeue()) - assertEquals(3, q.dequeue()) - assertNull(q.dequeue()) - } - - @Test - fun testRemoveHeadSegment() { - val q = SegmentBasedQueue<Int>() - q.enqueue(1) - val s = q.enqueue(2) - assertEquals(1, q.dequeue()) - q.enqueue(3) - s!!.removeSegment() - assertEquals(3, q.dequeue()) - assertNull(q.dequeue()) - } - - @Test - fun testClose() { - val q = SegmentBasedQueue<Int>() - q.enqueue(1) - assertEquals(0, q.close().id) - assertEquals(null, q.enqueue(2)) - assertEquals(1, q.dequeue()) - assertEquals(null, q.dequeue()) - } - - @Test - fun stressTest() { - val q = SegmentBasedQueue<Int>() - val expectedQueue = ArrayDeque<Int>() - val r = Random(0) - repeat(1_000_000 * stressTestMultiplier) { - if (r.nextBoolean()) { // add - val el = r.nextInt() - q.enqueue(el) - expectedQueue.add(el) - } else { // remove - assertEquals(expectedQueue.poll(), q.dequeue()) - q.checkHeadPrevIsCleaned() - } - } - } - - @Test - fun testRemoveSegmentsSerial() = stressTestRemoveSegments(false) - - @Test - fun testRemoveSegmentsRandom() = stressTestRemoveSegments(true) - - private fun stressTestRemoveSegments(random: Boolean) { - val N = 100_000 * stressTestMultiplier - val T = 10 - val q = SegmentBasedQueue<Int>() - val segments = (1..N).map { q.enqueue(it)!! }.toMutableList() - if (random) segments.shuffle() - assertEquals(N, q.numberOfSegments) - val nextSegmentIndex = AtomicInteger() - val barrier = CyclicBarrier(T) - (1..T).map { - thread { - barrier.await() - while (true) { - val i = nextSegmentIndex.getAndIncrement() - if (i >= N) break - segments[i].removeSegment() - } - } - }.forEach { it.join() } - assertEquals(2, q.numberOfSegments) - } -}
\ No newline at end of file diff --git a/integration/kotlinx-coroutines-jdk8/test/future/AsFutureTest.kt b/kotlinx-coroutines-core/jvm/test/jdk8/future/AsFutureTest.kt index 743816fa..743816fa 100644 --- a/integration/kotlinx-coroutines-jdk8/test/future/AsFutureTest.kt +++ b/kotlinx-coroutines-core/jvm/test/jdk8/future/AsFutureTest.kt diff --git a/integration/kotlinx-coroutines-jdk8/test/future/FutureAsDeferredUnhandledCompletionExceptionTest.kt b/kotlinx-coroutines-core/jvm/test/jdk8/future/FutureAsDeferredUnhandledCompletionExceptionTest.kt index bf810af7..9c9c97ec 100644 --- a/integration/kotlinx-coroutines-jdk8/test/future/FutureAsDeferredUnhandledCompletionExceptionTest.kt +++ b/kotlinx-coroutines-core/jvm/test/jdk8/future/FutureAsDeferredUnhandledCompletionExceptionTest.kt @@ -1,8 +1,8 @@ /* - * Copyright 2016-2021 JetBrains s.r.o. Use of this source code is governed by the Apache 2.0 license. + * Copyright 2016-2022 JetBrains s.r.o. Use of this source code is governed by the Apache 2.0 license. */ -package future +package kotlinx.coroutines.future import kotlinx.coroutines.* import kotlinx.coroutines.future.* diff --git a/integration/kotlinx-coroutines-jdk8/test/future/FutureExceptionsTest.kt b/kotlinx-coroutines-core/jvm/test/jdk8/future/FutureExceptionsTest.kt index 0c919b18..0c919b18 100644 --- a/integration/kotlinx-coroutines-jdk8/test/future/FutureExceptionsTest.kt +++ b/kotlinx-coroutines-core/jvm/test/jdk8/future/FutureExceptionsTest.kt diff --git a/integration/kotlinx-coroutines-jdk8/test/future/FutureTest.kt b/kotlinx-coroutines-core/jvm/test/jdk8/future/FutureTest.kt index 372e79ef..eda38165 100644 --- a/integration/kotlinx-coroutines-jdk8/test/future/FutureTest.kt +++ b/kotlinx-coroutines-core/jvm/test/jdk8/future/FutureTest.kt @@ -392,11 +392,7 @@ class FutureTest : TestBase() { } @Test - fun testUnhandledExceptionOnExternalCompletion() = runTest( - unhandled = listOf( - { it -> it is TestException } // exception is unhandled because there is no parent - ) - ) { + fun testUnhandledExceptionOnExternalCompletionIsNotReported() = runTest { expect(1) // No parent here (NonCancellable), so nowhere to propagate exception val result = future(NonCancellable + Dispatchers.Unconfined) { diff --git a/integration/kotlinx-coroutines-jdk8/test/stream/ConsumeAsFlowTest.kt b/kotlinx-coroutines-core/jvm/test/jdk8/stream/ConsumeAsFlowTest.kt index 96ff963d..96ff963d 100644 --- a/integration/kotlinx-coroutines-jdk8/test/stream/ConsumeAsFlowTest.kt +++ b/kotlinx-coroutines-core/jvm/test/jdk8/stream/ConsumeAsFlowTest.kt diff --git a/integration/kotlinx-coroutines-jdk8/test/time/DurationOverflowTest.kt b/kotlinx-coroutines-core/jvm/test/jdk8/time/DurationOverflowTest.kt index 9ab0ccf9..9ab0ccf9 100644 --- a/integration/kotlinx-coroutines-jdk8/test/time/DurationOverflowTest.kt +++ b/kotlinx-coroutines-core/jvm/test/jdk8/time/DurationOverflowTest.kt diff --git a/integration/kotlinx-coroutines-jdk8/test/time/FlowDebounceTest.kt b/kotlinx-coroutines-core/jvm/test/jdk8/time/FlowDebounceTest.kt index d6b2b8c1..d6b2b8c1 100644 --- a/integration/kotlinx-coroutines-jdk8/test/time/FlowDebounceTest.kt +++ b/kotlinx-coroutines-core/jvm/test/jdk8/time/FlowDebounceTest.kt diff --git a/integration/kotlinx-coroutines-jdk8/test/time/FlowSampleTest.kt b/kotlinx-coroutines-core/jvm/test/jdk8/time/FlowSampleTest.kt index d35ee72d..d35ee72d 100644 --- a/integration/kotlinx-coroutines-jdk8/test/time/FlowSampleTest.kt +++ b/kotlinx-coroutines-core/jvm/test/jdk8/time/FlowSampleTest.kt diff --git a/integration/kotlinx-coroutines-jdk8/test/time/WithTimeoutTest.kt b/kotlinx-coroutines-core/jvm/test/jdk8/time/WithTimeoutTest.kt index d33eaac1..d33eaac1 100644 --- a/integration/kotlinx-coroutines-jdk8/test/time/WithTimeoutTest.kt +++ b/kotlinx-coroutines-core/jvm/test/jdk8/time/WithTimeoutTest.kt diff --git a/kotlinx-coroutines-core/jvm/test/lincheck/ChannelsLincheckTest.kt b/kotlinx-coroutines-core/jvm/test/lincheck/ChannelsLincheckTest.kt index 74cc1783..092ef6fc 100644 --- a/kotlinx-coroutines-core/jvm/test/lincheck/ChannelsLincheckTest.kt +++ b/kotlinx-coroutines-core/jvm/test/lincheck/ChannelsLincheckTest.kt @@ -1,7 +1,7 @@ /* * Copyright 2016-2019 JetBrains s.r.o. Use of this source code is governed by the Apache 2.0 license. */ -@file:Suppress("unused") +@file:Suppress("unused", "MemberVisibilityCanBePrivate") package kotlinx.coroutines.lincheck @@ -15,109 +15,159 @@ import org.jetbrains.kotlinx.lincheck.* import org.jetbrains.kotlinx.lincheck.annotations.* import org.jetbrains.kotlinx.lincheck.annotations.Operation import org.jetbrains.kotlinx.lincheck.paramgen.* -import org.jetbrains.kotlinx.lincheck.verifier.* +import org.jetbrains.kotlinx.lincheck.strategy.managed.modelchecking.* -class RendezvousChannelLincheckTest : ChannelLincheckTestBase( +class RendezvousChannelLincheckTest : ChannelLincheckTestBaseWithOnSend( c = Channel(RENDEZVOUS), sequentialSpecification = SequentialRendezvousChannel::class.java ) class SequentialRendezvousChannel : SequentialIntChannelBase(RENDEZVOUS) -class Array1ChannelLincheckTest : ChannelLincheckTestBase( +class Buffered1ChannelLincheckTest : ChannelLincheckTestBaseWithOnSend( c = Channel(1), - sequentialSpecification = SequentialArray1RendezvousChannel::class.java + sequentialSpecification = SequentialBuffered1Channel::class.java ) -class SequentialArray1RendezvousChannel : SequentialIntChannelBase(1) +class Buffered1BroadcastChannelLincheckTest : ChannelLincheckTestBase( + c = ChannelViaBroadcast(BroadcastChannelImpl(1)), + sequentialSpecification = SequentialBuffered1Channel::class.java, + obstructionFree = false +) +class SequentialBuffered1Channel : SequentialIntChannelBase(1) -class Array2ChannelLincheckTest : ChannelLincheckTestBase( +class Buffered2ChannelLincheckTest : ChannelLincheckTestBaseWithOnSend( c = Channel(2), - sequentialSpecification = SequentialArray2RendezvousChannel::class.java + sequentialSpecification = SequentialBuffered2Channel::class.java +) +class Buffered2BroadcastChannelLincheckTest : ChannelLincheckTestBase( + c = ChannelViaBroadcast(BroadcastChannelImpl(2)), + sequentialSpecification = SequentialBuffered2Channel::class.java, + obstructionFree = false ) -class SequentialArray2RendezvousChannel : SequentialIntChannelBase(2) +class SequentialBuffered2Channel : SequentialIntChannelBase(2) -class UnlimitedChannelLincheckTest : ChannelLincheckTestBase( +class UnlimitedChannelLincheckTest : ChannelLincheckTestBaseAll( c = Channel(UNLIMITED), sequentialSpecification = SequentialUnlimitedChannel::class.java ) class SequentialUnlimitedChannel : SequentialIntChannelBase(UNLIMITED) -class ConflatedChannelLincheckTest : ChannelLincheckTestBase( +class ConflatedChannelLincheckTest : ChannelLincheckTestBaseAll( c = Channel(CONFLATED), - sequentialSpecification = SequentialConflatedChannel::class.java + sequentialSpecification = SequentialConflatedChannel::class.java, + obstructionFree = false +) +class ConflatedBroadcastChannelLincheckTest : ChannelLincheckTestBaseAll( + c = ChannelViaBroadcast(ConflatedBroadcastChannel()), + sequentialSpecification = SequentialConflatedChannel::class.java, + obstructionFree = false ) class SequentialConflatedChannel : SequentialIntChannelBase(CONFLATED) +abstract class ChannelLincheckTestBaseAll( + c: Channel<Int>, + sequentialSpecification: Class<*>, + obstructionFree: Boolean = true +) : ChannelLincheckTestBaseWithOnSend(c, sequentialSpecification, obstructionFree) { + @Operation + override fun trySend(value: Int) = super.trySend(value) + @Operation + override fun isClosedForReceive() = super.isClosedForReceive() + @Operation + override fun isEmpty() = super.isEmpty() +} + +abstract class ChannelLincheckTestBaseWithOnSend( + c: Channel<Int>, + sequentialSpecification: Class<*>, + obstructionFree: Boolean = true +) : ChannelLincheckTestBase(c, sequentialSpecification, obstructionFree) { + @Operation(allowExtraSuspension = true, blocking = true) + suspend fun sendViaSelect(@Param(name = "value") value: Int): Any = try { + select<Unit> { c.onSend(value) {} } + } catch (e: NumberedCancellationException) { + e.testResult + } +} + @Param.Params( - Param(name = "value", gen = IntGen::class, conf = "1:5"), - Param(name = "closeToken", gen = IntGen::class, conf = "1:3") + Param(name = "value", gen = IntGen::class, conf = "1:9"), + Param(name = "closeToken", gen = IntGen::class, conf = "1:9") ) abstract class ChannelLincheckTestBase( - private val c: Channel<Int>, - private val sequentialSpecification: Class<*> + protected val c: Channel<Int>, + private val sequentialSpecification: Class<*>, + private val obstructionFree: Boolean = true ) : AbstractLincheckTest() { - @Operation(promptCancellation = true) + + @Operation(allowExtraSuspension = true, blocking = true) suspend fun send(@Param(name = "value") value: Int): Any = try { c.send(value) } catch (e: NumberedCancellationException) { e.testResult } - @Operation - fun trySend(@Param(name = "value") value: Int): Any = c.trySend(value) - .onSuccess { return true } - .onFailure { - return if (it is NumberedCancellationException) it.testResult - else false - } - - // TODO: this operation should be (and can be!) linearizable, but is not - // @Operation - suspend fun sendViaSelect(@Param(name = "value") value: Int): Any = try { - select<Unit> { c.onSend(value) {} } - } catch (e: NumberedCancellationException) { - e.testResult - } + // @Operation TODO: `trySend()` is not linearizable as it can fail due to postponed buffer expansion + // TODO: or make a rendezvous with `tryReceive`, which violates the sequential specification. + open fun trySend(@Param(name = "value") value: Int): Any = c.trySend(value) + .onSuccess { return true } + .onFailure { + return if (it is NumberedCancellationException) it.testResult + else false + } - @Operation(promptCancellation = true) + @Operation(allowExtraSuspension = true, blocking = true) suspend fun receive(): Any = try { c.receive() } catch (e: NumberedCancellationException) { e.testResult } - @Operation + @Operation(allowExtraSuspension = true, blocking = true) + suspend fun receiveCatching(): Any = c.receiveCatching() + .onSuccess { return it } + .onClosed { e -> return (e as NumberedCancellationException).testResult } + + @Operation(blocking = true) fun tryReceive(): Any? = c.tryReceive() .onSuccess { return it } .onFailure { return if (it is NumberedCancellationException) it.testResult else null } - // TODO: this operation should be (and can be!) linearizable, but is not - // @Operation + @Operation(allowExtraSuspension = true, blocking = true) suspend fun receiveViaSelect(): Any = try { select<Int> { c.onReceive { it } } } catch (e: NumberedCancellationException) { e.testResult } - @Operation(causesBlocking = true) + @Operation(causesBlocking = true, blocking = true) fun close(@Param(name = "closeToken") token: Int): Boolean = c.close(NumberedCancellationException(token)) - // TODO: this operation should be (and can be!) linearizable, but is not - // @Operation + @Operation(causesBlocking = true, blocking = true) fun cancel(@Param(name = "closeToken") token: Int) = c.cancel(NumberedCancellationException(token)) - // @Operation - fun isClosedForReceive() = c.isClosedForReceive + // @Operation TODO non-linearizable in BufferedChannel + open fun isClosedForReceive() = c.isClosedForReceive - // @Operation + @Operation(blocking = true) fun isClosedForSend() = c.isClosedForSend - // TODO: this operation should be (and can be!) linearizable, but is not - // @Operation - fun isEmpty() = c.isEmpty + // @Operation TODO non-linearizable in BufferedChannel + open fun isEmpty() = c.isEmpty + + @StateRepresentation + fun state() = (c as? BufferedChannel<*>)?.toStringDebug() ?: c.toString() + + @Validate + fun validate() { + (c as? BufferedChannel<*>)?.checkSegmentStructureInvariants() + } - override fun <O : Options<O, *>> O.customize(isStressTest: Boolean): O = + override fun <O : Options<O, *>> O.customize(isStressTest: Boolean) = actorsBefore(0).sequentialSpecification(sequentialSpecification) + + override fun ModelCheckingOptions.customize(isStressTest: Boolean) = + checkObstructionFreedom(obstructionFree) } private class NumberedCancellationException(number: Int) : CancellationException() { @@ -125,7 +175,7 @@ private class NumberedCancellationException(number: Int) : CancellationException } -abstract class SequentialIntChannelBase(private val capacity: Int) : VerifierState() { +abstract class SequentialIntChannelBase(private val capacity: Int) { private val senders = ArrayList<Pair<CancellableContinuation<Any>, Int>>() private val receivers = ArrayList<CancellableContinuation<Any>>() private val buffer = ArrayList<Int>() @@ -167,6 +217,8 @@ abstract class SequentialIntChannelBase(private val capacity: Int) : VerifierSta receivers.add(cont) } + suspend fun receiveCatching() = receive() + fun tryReceive(): Any? { if (buffer.isNotEmpty()) { val el = buffer.removeAt(0) @@ -200,7 +252,7 @@ abstract class SequentialIntChannelBase(private val capacity: Int) : VerifierSta } fun cancel(token: Int) { - if (!close(token)) return + close(token) for ((s, _) in senders) s.resume(closedMessage!!) senders.clear() buffer.clear() @@ -213,8 +265,6 @@ abstract class SequentialIntChannelBase(private val capacity: Int) : VerifierSta if (closedMessage !== null) return false return buffer.isEmpty() && senders.isEmpty() } - - override fun extractState() = buffer to closedMessage } private fun <T> CancellableContinuation<T>.resume(res: T): Boolean { diff --git a/kotlinx-coroutines-core/jvm/test/lincheck/LockFreeListLincheckTest.kt b/kotlinx-coroutines-core/jvm/test/lincheck/LockFreeListLincheckTest.kt deleted file mode 100644 index 4f1bb6ad..00000000 --- a/kotlinx-coroutines-core/jvm/test/lincheck/LockFreeListLincheckTest.kt +++ /dev/null @@ -1,53 +0,0 @@ -/* - * Copyright 2016-2019 JetBrains s.r.o. Use of this source code is governed by the Apache 2.0 license. - */ -@file:Suppress("unused") - -package kotlinx.coroutines.lincheck - -import kotlinx.coroutines.* -import kotlinx.coroutines.internal.* -import org.jetbrains.kotlinx.lincheck.annotations.* -import org.jetbrains.kotlinx.lincheck.annotations.Operation -import org.jetbrains.kotlinx.lincheck.paramgen.* -import org.jetbrains.kotlinx.lincheck.strategy.managed.modelchecking.* - -@Param(name = "value", gen = IntGen::class, conf = "1:5") -class LockFreeListLincheckTest : AbstractLincheckTest() { - class Node(val value: Int): LockFreeLinkedListNode() - - private val q: LockFreeLinkedListHead = LockFreeLinkedListHead() - - @Operation - fun addLast(@Param(name = "value") value: Int) { - q.addLast(Node(value)) - } - - @Operation - fun addLastIfNotSame(@Param(name = "value") value: Int) { - q.addLastIfPrev(Node(value)) { !it.isSame(value) } - } - - @Operation - fun removeFirst(): Int? { - val node = q.removeFirstOrNull() ?: return null - return (node as Node).value - } - - @Operation - fun removeFirstOrPeekIfNotSame(@Param(name = "value") value: Int): Int? { - val node = q.removeFirstIfIsInstanceOfOrPeekIf<Node> { !it.isSame(value) } ?: return null - return node.value - } - - private fun Any.isSame(value: Int) = this is Node && this.value == value - - override fun extractState(): Any { - val elements = ArrayList<Int>() - q.forEach<Node> { elements.add(it.value) } - return elements - } - - override fun ModelCheckingOptions.customize(isStressTest: Boolean) = - checkObstructionFreedom() -}
\ No newline at end of file diff --git a/kotlinx-coroutines-core/jvm/test/lincheck/LockFreeTaskQueueLincheckTest.kt b/kotlinx-coroutines-core/jvm/test/lincheck/LockFreeTaskQueueLincheckTest.kt index 2a9164e1..11f5535b 100644 --- a/kotlinx-coroutines-core/jvm/test/lincheck/LockFreeTaskQueueLincheckTest.kt +++ b/kotlinx-coroutines-core/jvm/test/lincheck/LockFreeTaskQueueLincheckTest.kt @@ -30,8 +30,6 @@ internal abstract class AbstractLockFreeTaskQueueWithoutRemoveLincheckTest( override fun <O : Options<O, *>> O.customize(isStressTest: Boolean): O = verifier(QuiescentConsistencyVerifier::class.java) - override fun extractState() = q.map { it } to q.isClosed() - override fun ModelCheckingOptions.customize(isStressTest: Boolean) = checkObstructionFreedom() } @@ -42,9 +40,8 @@ internal class MCLockFreeTaskQueueWithRemoveLincheckTest : AbstractLockFreeTaskQ fun removeFirstOrNull() = q.removeFirstOrNull() } -@OpGroupConfig(name = "consumer", nonParallel = true) internal class SCLockFreeTaskQueueWithRemoveLincheckTest : AbstractLockFreeTaskQueueWithoutRemoveLincheckTest(singleConsumer = true) { @QuiescentConsistent - @Operation(group = "consumer") + @Operation(nonParallelGroup = "consumer") fun removeFirstOrNull() = q.removeFirstOrNull() }
\ No newline at end of file diff --git a/kotlinx-coroutines-core/jvm/test/lincheck/MutexLincheckTest.kt b/kotlinx-coroutines-core/jvm/test/lincheck/MutexLincheckTest.kt index a278985f..983a64ac 100644 --- a/kotlinx-coroutines-core/jvm/test/lincheck/MutexLincheckTest.kt +++ b/kotlinx-coroutines-core/jvm/test/lincheck/MutexLincheckTest.kt @@ -5,28 +5,40 @@ package kotlinx.coroutines.lincheck import kotlinx.coroutines.* +import kotlinx.coroutines.selects.* import kotlinx.coroutines.sync.* import org.jetbrains.kotlinx.lincheck.* +import org.jetbrains.kotlinx.lincheck.annotations.* import org.jetbrains.kotlinx.lincheck.annotations.Operation -import org.jetbrains.kotlinx.lincheck.strategy.managed.modelchecking.* +import org.jetbrains.kotlinx.lincheck.paramgen.* +@Param(name = "owner", gen = IntGen::class, conf = "0:2") class MutexLincheckTest : AbstractLincheckTest() { private val mutex = Mutex() - @Operation - fun tryLock() = mutex.tryLock() + @Operation(handleExceptionsAsResult = [IllegalStateException::class]) + fun tryLock(@Param(name = "owner") owner: Int) = mutex.tryLock(owner.asOwnerOrNull) + // TODO: `lock()` with non-null owner is non-linearizable @Operation(promptCancellation = true) - suspend fun lock() = mutex.lock() + suspend fun lock() = mutex.lock(null) + + // TODO: `onLock` with non-null owner is non-linearizable + // onLock may suspend in case of clause re-registration. + @Operation(allowExtraSuspension = true, promptCancellation = true) + suspend fun onLock() = select<Unit> { mutex.onLock(null) {} } @Operation(handleExceptionsAsResult = [IllegalStateException::class]) - fun unlock() = mutex.unlock() + fun unlock(@Param(name = "owner") owner: Int) = mutex.unlock(owner.asOwnerOrNull) + + @Operation + fun isLocked() = mutex.isLocked + + @Operation + fun holdsLock(@Param(name = "owner") owner: Int) = mutex.holdsLock(owner) override fun <O : Options<O, *>> O.customize(isStressTest: Boolean): O = actorsBefore(0) - override fun ModelCheckingOptions.customize(isStressTest: Boolean) = - checkObstructionFreedom() - - override fun extractState() = mutex.isLocked + private val Int.asOwnerOrNull get() = if (this == 0) null else this } diff --git a/kotlinx-coroutines-core/jvm/test/lincheck/ResizableAtomicArrayLincheckTest.kt b/kotlinx-coroutines-core/jvm/test/lincheck/ResizableAtomicArrayLincheckTest.kt index 1948a78e..e937b37e 100644 --- a/kotlinx-coroutines-core/jvm/test/lincheck/ResizableAtomicArrayLincheckTest.kt +++ b/kotlinx-coroutines-core/jvm/test/lincheck/ResizableAtomicArrayLincheckTest.kt @@ -11,17 +11,14 @@ import org.jetbrains.kotlinx.lincheck.paramgen.* @Param(name = "index", gen = IntGen::class, conf = "0:4") @Param(name = "value", gen = IntGen::class, conf = "1:5") -@OpGroupConfig(name = "sync", nonParallel = true) class ResizableAtomicArrayLincheckTest : AbstractLincheckTest() { private val a = ResizableAtomicArray<Int>(2) @Operation fun get(@Param(name = "index") index: Int): Int? = a[index] - @Operation(group = "sync") + @Operation(nonParallelGroup = "writer") fun set(@Param(name = "index") index: Int, @Param(name = "value") value: Int) { a.setSynchronized(index, value) } - - override fun extractState() = (0..4).map { a[it] } }
\ No newline at end of file diff --git a/kotlinx-coroutines-core/jvm/test/lincheck/SegmentListRemoveLincheckTest.kt b/kotlinx-coroutines-core/jvm/test/lincheck/SegmentListRemoveLincheckTest.kt deleted file mode 100644 index 5a8d7b47..00000000 --- a/kotlinx-coroutines-core/jvm/test/lincheck/SegmentListRemoveLincheckTest.kt +++ /dev/null @@ -1,43 +0,0 @@ -/* - * Copyright 2016-2020 JetBrains s.r.o. Use of this source code is governed by the Apache 2.0 license. - */ - -@file:Suppress("unused") - -package kotlinx.coroutines.lincheck - -import kotlinx.coroutines.* -import kotlinx.coroutines.internal.* -import org.jetbrains.kotlinx.lincheck.* -import org.jetbrains.kotlinx.lincheck.annotations.* -import org.jetbrains.kotlinx.lincheck.paramgen.* -import org.jetbrains.kotlinx.lincheck.strategy.managed.modelchecking.* - -class SegmentListRemoveLincheckTest : AbstractLincheckTest() { - private val q = SegmentBasedQueue<Int>() - private val segments: Array<OneElementSegment<Int>> - - init { - segments = (0..5).map { q.enqueue(it)!! }.toTypedArray() - q.enqueue(6) - } - - @Operation - fun removeSegment(@Param(gen = IntGen::class, conf = "1:5") index: Int) { - segments[index].removeSegment() - } - - override fun <O : Options<O, *>> O.customize(isStressTest: Boolean): O = this - .actorsBefore(0).actorsAfter(0) - - override fun extractState() = segments.map { it.logicallyRemoved } - - @Validate - fun checkAllRemoved() { - q.checkHeadPrevIsCleaned() - q.checkAllSegmentsAreNotLogicallyRemoved() - } - - override fun ModelCheckingOptions.customize(isStressTest: Boolean) = - checkObstructionFreedom() -}
\ No newline at end of file diff --git a/kotlinx-coroutines-core/jvm/test/lincheck/SegmentQueueLincheckTest.kt b/kotlinx-coroutines-core/jvm/test/lincheck/SegmentQueueLincheckTest.kt deleted file mode 100644 index 76a59e39..00000000 --- a/kotlinx-coroutines-core/jvm/test/lincheck/SegmentQueueLincheckTest.kt +++ /dev/null @@ -1,44 +0,0 @@ -/* - * Copyright 2016-2019 JetBrains s.r.o. Use of this source code is governed by the Apache 2.0 license. - */ -@file:Suppress("unused") - -package kotlinx.coroutines.lincheck - -import kotlinx.coroutines.* -import kotlinx.coroutines.internal.SegmentBasedQueue -import org.jetbrains.kotlinx.lincheck.annotations.* -import org.jetbrains.kotlinx.lincheck.annotations.Operation -import org.jetbrains.kotlinx.lincheck.paramgen.* -import org.jetbrains.kotlinx.lincheck.strategy.managed.modelchecking.* - -@Param(name = "value", gen = IntGen::class, conf = "1:5") -class SegmentQueueLincheckTest : AbstractLincheckTest() { - private val q = SegmentBasedQueue<Int>() - - @Operation - fun enqueue(@Param(name = "value") x: Int): Boolean { - return q.enqueue(x) !== null - } - - @Operation - fun dequeue(): Int? = q.dequeue() - - @Operation - fun close() { - q.close() - } - - override fun extractState(): Any { - val elements = ArrayList<Int>() - while (true) { - val x = q.dequeue() ?: break - elements.add(x) - } - val closed = q.enqueue(0) === null - return elements to closed - } - - override fun ModelCheckingOptions.customize(isStressTest: Boolean) = - checkObstructionFreedom() -}
\ No newline at end of file diff --git a/kotlinx-coroutines-core/jvm/test/lincheck/SemaphoreLincheckTest.kt b/kotlinx-coroutines-core/jvm/test/lincheck/SemaphoreLincheckTest.kt index 2b471d7f..d093e806 100644 --- a/kotlinx-coroutines-core/jvm/test/lincheck/SemaphoreLincheckTest.kt +++ b/kotlinx-coroutines-core/jvm/test/lincheck/SemaphoreLincheckTest.kt @@ -11,7 +11,7 @@ import org.jetbrains.kotlinx.lincheck.annotations.Operation import org.jetbrains.kotlinx.lincheck.strategy.managed.modelchecking.* abstract class SemaphoreLincheckTestBase(permits: Int) : AbstractLincheckTest() { - private val semaphore = Semaphore(permits) + private val semaphore = SemaphoreImpl(permits = permits, acquiredPermits = 0) @Operation fun tryAcquire() = semaphore.tryAcquire() @@ -25,11 +25,9 @@ abstract class SemaphoreLincheckTestBase(permits: Int) : AbstractLincheckTest() override fun <O : Options<O, *>> O.customize(isStressTest: Boolean): O = actorsBefore(0) - override fun extractState() = semaphore.availablePermits - override fun ModelCheckingOptions.customize(isStressTest: Boolean) = checkObstructionFreedom() } class Semaphore1LincheckTest : SemaphoreLincheckTestBase(1) -class Semaphore2LincheckTest : SemaphoreLincheckTestBase(2)
\ No newline at end of file +class Semaphore2LincheckTest : SemaphoreLincheckTestBase(2) diff --git a/kotlinx-coroutines-core/jvm/test/scheduling/CoroutineSchedulerInternalApiStressTest.kt b/kotlinx-coroutines-core/jvm/test/scheduling/CoroutineSchedulerInternalApiStressTest.kt new file mode 100644 index 00000000..2ec714ff --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/scheduling/CoroutineSchedulerInternalApiStressTest.kt @@ -0,0 +1,89 @@ +/* + * Copyright 2016-2023 JetBrains s.r.o. Use of this source code is governed by the Apache 2.0 license. + */ + +package kotlinx.coroutines.scheduling + +import kotlinx.coroutines.* +import kotlinx.coroutines.internal.AVAILABLE_PROCESSORS +import org.junit.Test +import java.util.* +import java.util.concurrent.ConcurrentHashMap +import java.util.concurrent.CountDownLatch +import java.util.concurrent.CyclicBarrier +import java.util.concurrent.atomic.AtomicInteger +import kotlin.random.* +import kotlin.random.Random +import kotlin.test.* +import kotlin.time.* + +class CoroutineSchedulerInternalApiStressTest : TestBase() { + + @Test(timeout = 120_000L) + fun testHelpDefaultIoIsIsolated() = repeat(100 * stressTestMultiplierSqrt) { + val ioTaskMarker = ThreadLocal.withInitial { false } + runTest { + val jobToComplete = Job() + val expectedIterations = 100 + val completionLatch = CountDownLatch(1) + val tasksToCompleteJob = AtomicInteger(expectedIterations) + val observedIoThreads = Collections.newSetFromMap(ConcurrentHashMap<Thread, Boolean>()) + val observedDefaultThreads = Collections.newSetFromMap(ConcurrentHashMap<Thread, Boolean>()) + + val barrier = CyclicBarrier(AVAILABLE_PROCESSORS) + val spawners = ArrayList<Job>() + repeat(AVAILABLE_PROCESSORS - 1) { + // Launch CORES - 1 spawners + spawners += launch(Dispatchers.Default) { + barrier.await() + repeat(expectedIterations) { + launch { + val tasksLeft = tasksToCompleteJob.decrementAndGet() + if (tasksLeft < 0) return@launch // Leftovers are being executed all over the place + observedDefaultThreads.add(Thread.currentThread()) + if (tasksLeft == 0) { + // Verify threads first + try { + assertFalse(observedIoThreads.containsAll(observedDefaultThreads)) + } finally { + jobToComplete.complete() + } + } + } + + // Sometimes launch an IO task to mess with a scheduler + if (Random.nextInt(0..9) == 0) { + launch(Dispatchers.IO) { + ioTaskMarker.set(true) + observedIoThreads.add(Thread.currentThread()) + assertTrue(Thread.currentThread().isIoDispatcherThread()) + } + } + } + completionLatch.await() + } + } + + withContext(Dispatchers.Default) { + barrier.await() + var timesHelped = 0 + while (!jobToComplete.isCompleted) { + val result = runSingleTaskFromCurrentSystemDispatcher() + assertFalse(ioTaskMarker.get()) + if (result == 0L) { + ++timesHelped + continue + } else if (result >= 0L) { + Thread.sleep(result.toDuration(DurationUnit.NANOSECONDS).toDelayMillis()) + } else { + Thread.sleep(10) + } + } + completionLatch.countDown() + assertEquals(100, timesHelped) + assertTrue(Thread.currentThread() in observedDefaultThreads, observedDefaultThreads.toString()) + } + } + } +} + diff --git a/kotlinx-coroutines-core/jvm/test/scheduling/CoroutineSchedulerOversubscriptionTest.kt b/kotlinx-coroutines-core/jvm/test/scheduling/CoroutineSchedulerOversubscriptionTest.kt new file mode 100644 index 00000000..0fd6159f --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/scheduling/CoroutineSchedulerOversubscriptionTest.kt @@ -0,0 +1,89 @@ +/* + * Copyright 2016-2022 JetBrains s.r.o. Use of this source code is governed by the Apache 2.0 license. + */ + +package kotlinx.coroutines.scheduling + +import kotlinx.coroutines.* +import org.junit.Test +import java.util.concurrent.* +import java.util.concurrent.atomic.AtomicInteger + +class CoroutineSchedulerOversubscriptionTest : TestBase() { + + private val inDefault = AtomicInteger(0) + + private fun CountDownLatch.runAndCheck() { + if (inDefault.incrementAndGet() > CORE_POOL_SIZE) { + error("Oversubscription detected") + } + + await() + inDefault.decrementAndGet() + } + + @Test + fun testOverSubscriptionDeterministic() = runTest { + val barrier = CountDownLatch(1) + val threadsOccupiedBarrier = CyclicBarrier(CORE_POOL_SIZE) + // All threads but one + repeat(CORE_POOL_SIZE - 1) { + launch(Dispatchers.Default) { + threadsOccupiedBarrier.await() + barrier.runAndCheck() + } + } + threadsOccupiedBarrier.await() + withContext(Dispatchers.Default) { + // Put a task in a local queue, it will be stolen + launch(Dispatchers.Default) { + barrier.runAndCheck() + } + // Put one more task to trick the local queue check + launch(Dispatchers.Default) { + barrier.runAndCheck() + } + + withContext(Dispatchers.IO) { + try { + // Release the thread + delay(100) + } finally { + barrier.countDown() + } + } + } + } + + @Test + fun testOverSubscriptionStress() = repeat(1000 * stressTestMultiplierSqrt) { + inDefault.set(0) + runTest { + val barrier = CountDownLatch(1) + val threadsOccupiedBarrier = CyclicBarrier(CORE_POOL_SIZE) + // All threads but one + repeat(CORE_POOL_SIZE - 1) { + launch(Dispatchers.Default) { + threadsOccupiedBarrier.await() + barrier.runAndCheck() + } + } + threadsOccupiedBarrier.await() + withContext(Dispatchers.Default) { + // Put a task in a local queue + launch(Dispatchers.Default) { + barrier.runAndCheck() + } + // Put one more task to trick the local queue check + launch(Dispatchers.Default) { + barrier.runAndCheck() + } + + withContext(Dispatchers.IO) { + yield() + barrier.countDown() + } + } + } + } +} diff --git a/kotlinx-coroutines-core/jvm/test/scheduling/WorkQueueStressTest.kt b/kotlinx-coroutines-core/jvm/test/scheduling/WorkQueueStressTest.kt index 5e170c9f..e2562b57 100644 --- a/kotlinx-coroutines-core/jvm/test/scheduling/WorkQueueStressTest.kt +++ b/kotlinx-coroutines-core/jvm/test/scheduling/WorkQueueStressTest.kt @@ -9,6 +9,7 @@ import org.junit.* import org.junit.Test import java.util.concurrent.* import kotlin.concurrent.* +import kotlin.jvm.internal.* import kotlin.test.* class WorkQueueStressTest : TestBase() { @@ -40,7 +41,7 @@ class WorkQueueStressTest : TestBase() { threads += thread(name = "producer") { startLatch.await() for (i in 1..offerIterations) { - while (producerQueue.bufferSize > BUFFER_CAPACITY / 2) { + while (producerQueue.size > BUFFER_CAPACITY / 2) { Thread.yield() } @@ -52,17 +53,18 @@ class WorkQueueStressTest : TestBase() { for (i in 0 until stealersCount) { threads += thread(name = "stealer $i") { + val ref = Ref.ObjectRef<Task?>() val myQueue = WorkQueue() startLatch.await() while (!producerFinished || producerQueue.size != 0) { - stolenTasks[i].addAll(myQueue.drain().map { task(it) }) - myQueue.tryStealFrom(victim = producerQueue) + stolenTasks[i].addAll(myQueue.drain(ref).map { task(it) }) + producerQueue.trySteal(ref) } // Drain last element which is not counted in buffer - stolenTasks[i].addAll(myQueue.drain().map { task(it) }) - myQueue.tryStealFrom(producerQueue) - stolenTasks[i].addAll(myQueue.drain().map { task(it) }) + stolenTasks[i].addAll(myQueue.drain(ref).map { task(it) }) + producerQueue.trySteal(ref) + stolenTasks[i].addAll(myQueue.drain(ref).map { task(it) }) } } @@ -77,7 +79,7 @@ class WorkQueueStressTest : TestBase() { threads += thread(name = "producer") { startLatch.await() for (i in 1..offerIterations) { - while (producerQueue.bufferSize == BUFFER_CAPACITY - 1) { + while (producerQueue.size == BUFFER_CAPACITY - 1) { Thread.yield() } @@ -89,13 +91,14 @@ class WorkQueueStressTest : TestBase() { val stolen = GlobalQueue() threads += thread(name = "stealer") { val myQueue = WorkQueue() + val ref = Ref.ObjectRef<Task?>() startLatch.await() while (stolen.size != offerIterations) { - if (myQueue.tryStealFrom(producerQueue) != NOTHING_TO_STEAL) { - stolen.addAll(myQueue.drain().map { task(it) }) + if (producerQueue.trySteal(ref) != NOTHING_TO_STEAL) { + stolen.addAll(myQueue.drain(ref).map { task(it) }) } } - stolen.addAll(myQueue.drain().map { task(it) }) + stolen.addAll(myQueue.drain(ref).map { task(it) }) } startLatch.countDown() diff --git a/kotlinx-coroutines-core/jvm/test/scheduling/WorkQueueTest.kt b/kotlinx-coroutines-core/jvm/test/scheduling/WorkQueueTest.kt index 7acd1620..f690d388 100644 --- a/kotlinx-coroutines-core/jvm/test/scheduling/WorkQueueTest.kt +++ b/kotlinx-coroutines-core/jvm/test/scheduling/WorkQueueTest.kt @@ -7,6 +7,7 @@ package kotlinx.coroutines.scheduling import kotlinx.coroutines.* import org.junit.* import org.junit.Test +import kotlin.jvm.internal.Ref.ObjectRef import kotlin.test.* class WorkQueueTest : TestBase() { @@ -27,7 +28,7 @@ class WorkQueueTest : TestBase() { fun testLastScheduledComesFirst() { val queue = WorkQueue() (1L..4L).forEach { queue.add(task(it)) } - assertEquals(listOf(4L, 1L, 2L, 3L), queue.drain()) + assertEquals(listOf(4L, 1L, 2L, 3L), queue.drain(ObjectRef())) } @Test @@ -38,9 +39,9 @@ class WorkQueueTest : TestBase() { (0 until size).forEach { queue.add(task(it))?.let { t -> offload.addLast(t) } } val expectedResult = listOf(129L) + (0L..126L).toList() - val actualResult = queue.drain() + val actualResult = queue.drain(ObjectRef()) assertEquals(expectedResult, actualResult) - assertEquals((0L until size).toSet().minus(expectedResult), offload.drain().toSet()) + assertEquals((0L until size).toSet().minus(expectedResult.toSet()), offload.drain().toSet()) } @Test @@ -61,23 +62,39 @@ class WorkQueueTest : TestBase() { timeSource.step(3) val stealer = WorkQueue() - assertEquals(TASK_STOLEN, stealer.tryStealFrom(victim)) - assertEquals(arrayListOf(1L), stealer.drain()) + val ref = ObjectRef<Task?>() + assertEquals(TASK_STOLEN, victim.trySteal(ref)) + assertEquals(arrayListOf(1L), stealer.drain(ref)) - assertEquals(TASK_STOLEN, stealer.tryStealFrom(victim)) - assertEquals(arrayListOf(2L), stealer.drain()) + assertEquals(TASK_STOLEN, victim.trySteal(ref)) + assertEquals(arrayListOf(2L), stealer.drain(ref)) + } + + @Test + fun testPollBlocking() { + val queue = WorkQueue() + assertNull(queue.pollBlocking()) + val blockingTask = blockingTask(1L) + queue.add(blockingTask) + queue.add(task(1L)) + assertSame(blockingTask, queue.pollBlocking()) } } internal fun task(n: Long) = TaskImpl(Runnable {}, n, NonBlockingContext) +internal fun blockingTask(n: Long) = TaskImpl(Runnable {}, n, BlockingContext) -internal fun WorkQueue.drain(): List<Long> { +internal fun WorkQueue.drain(ref: ObjectRef<Task?>): List<Long> { var task: Task? = poll() val result = arrayListOf<Long>() while (task != null) { result += task.submissionTime task = poll() } + if (ref.element != null) { + result += ref.element!!.submissionTime + ref.element = null + } return result } @@ -90,3 +107,5 @@ internal fun GlobalQueue.drain(): List<Long> { } return result } + +internal fun WorkQueue.trySteal(stolenTaskRef: ObjectRef<Task?>): Long = trySteal(STEAL_ANY, stolenTaskRef) diff --git a/kotlinx-coroutines-core/jvm/test/selects/SelectMemoryLeakStressTest.kt b/kotlinx-coroutines-core/jvm/test/selects/SelectMemoryLeakStressTest.kt index 7f924dba..2ddf133f 100644 --- a/kotlinx-coroutines-core/jvm/test/selects/SelectMemoryLeakStressTest.kt +++ b/kotlinx-coroutines-core/jvm/test/selects/SelectMemoryLeakStressTest.kt @@ -40,7 +40,7 @@ class SelectMemoryLeakStressTest : TestBase() { val data = Channel<Int>(1) repeat(nRepeat) { value -> val bigValue = bigValue() // new instance - select<Unit> { + select { leak.onReceive { println("Capture big value into this lambda: $bigValue") expectUnreached() diff --git a/kotlinx-coroutines-core/native/src/Builders.kt b/kotlinx-coroutines-core/native/src/Builders.kt index f5b22224..1f1d352d 100644 --- a/kotlinx-coroutines-core/native/src/Builders.kt +++ b/kotlinx-coroutines-core/native/src/Builders.kt @@ -52,8 +52,45 @@ public actual fun <T> runBlocking(context: CoroutineContext, block: suspend Coro newContext = GlobalScope.newCoroutineContext(context) } val coroutine = BlockingCoroutine<T>(newContext, eventLoop) - coroutine.start(CoroutineStart.DEFAULT, coroutine, block) - return coroutine.joinBlocking() + var completed = false + ThreadLocalKeepAlive.addCheck { !completed } + try { + coroutine.start(CoroutineStart.DEFAULT, coroutine, block) + return coroutine.joinBlocking() + } finally { + completed = true + } +} + +@ThreadLocal +private object ThreadLocalKeepAlive { + /** If any of these checks passes, this means this [Worker] is still used. */ + private var checks = mutableListOf<() -> Boolean>() + + /** Whether the worker currently tries to keep itself alive. */ + private var keepAliveLoopActive = false + + /** Adds another stopgap that must be passed before the [Worker] can be terminated. */ + fun addCheck(terminationForbidden: () -> Boolean) { + checks.add(terminationForbidden) + if (!keepAliveLoopActive) keepAlive() + } + + /** + * Send a ping to the worker to prevent it from terminating while this coroutine is running, + * ensuring that continuations don't get dropped and forgotten. + */ + private fun keepAlive() { + // only keep the checks that still forbid the termination + checks = checks.filter { it() }.toMutableList() + // if there are no checks left, we no longer keep the worker alive, it can be terminated + keepAliveLoopActive = checks.isNotEmpty() + if (keepAliveLoopActive) { + Worker.current.executeAfter(afterMicroseconds = 100_000) { + keepAlive() + } + } + } } private class BlockingCoroutine<T>( diff --git a/kotlinx-coroutines-core/native/src/CoroutineContext.kt b/kotlinx-coroutines-core/native/src/CoroutineContext.kt index 6e2dac1a..51ffd731 100644 --- a/kotlinx-coroutines-core/native/src/CoroutineContext.kt +++ b/kotlinx-coroutines-core/native/src/CoroutineContext.kt @@ -6,42 +6,31 @@ package kotlinx.coroutines import kotlinx.coroutines.internal.* import kotlin.coroutines.* -import kotlin.native.concurrent.* internal actual object DefaultExecutor : CoroutineDispatcher(), Delay { private val delegate = WorkerDispatcher(name = "DefaultExecutor") override fun dispatch(context: CoroutineContext, block: Runnable) { - checkState() delegate.dispatch(context, block) } override fun scheduleResumeAfterDelay(timeMillis: Long, continuation: CancellableContinuation<Unit>) { - checkState() delegate.scheduleResumeAfterDelay(timeMillis, continuation) } override fun invokeOnTimeout(timeMillis: Long, block: Runnable, context: CoroutineContext): DisposableHandle { - checkState() return delegate.invokeOnTimeout(timeMillis, block, context) } actual fun enqueue(task: Runnable): Unit { - checkState() delegate.dispatch(EmptyCoroutineContext, task) } - - private fun checkState() { - if (multithreadingSupported) return - error("DefaultExecutor should never be invoked in K/N with disabled new memory model. The top-most 'runBlocking' event loop has been shutdown") - } } internal expect fun createDefaultDispatcher(): CoroutineDispatcher -@SharedImmutable -internal actual val DefaultDelay: Delay = if (multithreadingSupported) DefaultExecutor else OldDefaultExecutor +internal actual val DefaultDelay: Delay = DefaultExecutor public actual fun CoroutineScope.newCoroutineContext(context: CoroutineContext): CoroutineContext { val combined = coroutineContext + context diff --git a/kotlinx-coroutines-core/native/src/CoroutineExceptionHandlerImpl.kt b/kotlinx-coroutines-core/native/src/CoroutineExceptionHandlerImpl.kt deleted file mode 100644 index 434813dc..00000000 --- a/kotlinx-coroutines-core/native/src/CoroutineExceptionHandlerImpl.kt +++ /dev/null @@ -1,14 +0,0 @@ -/* - * Copyright 2016-2021 JetBrains s.r.o. Use of this source code is governed by the Apache 2.0 license. - */ - -package kotlinx.coroutines - -import kotlin.coroutines.* -import kotlin.native.* - -@OptIn(ExperimentalStdlibApi::class) -internal actual fun handleCoroutineExceptionImpl(context: CoroutineContext, exception: Throwable) { - // log exception - processUnhandledException(exception) -} diff --git a/kotlinx-coroutines-core/native/src/Dispatchers.kt b/kotlinx-coroutines-core/native/src/Dispatchers.kt index 6c51a034..2576ba5c 100644 --- a/kotlinx-coroutines-core/native/src/Dispatchers.kt +++ b/kotlinx-coroutines-core/native/src/Dispatchers.kt @@ -4,11 +4,12 @@ package kotlinx.coroutines -import kotlinx.coroutines.internal.multithreadingSupported +import kotlinx.coroutines.internal.* import kotlin.coroutines.* + public actual object Dispatchers { - public actual val Default: CoroutineDispatcher = createDefaultDispatcherBasedOnMm() + public actual val Default: CoroutineDispatcher = createDefaultDispatcher() public actual val Main: MainCoroutineDispatcher get() = injectedMainDispatcher ?: mainDispatcher public actual val Unconfined: CoroutineDispatcher get() = kotlinx.coroutines.Unconfined // Avoid freezing @@ -19,43 +20,37 @@ public actual object Dispatchers { @PublishedApi internal fun injectMain(dispatcher: MainCoroutineDispatcher) { - if (!multithreadingSupported) { - throw IllegalStateException("Dispatchers.setMain is not supported in Kotlin/Native when new memory model is disabled") - } injectedMainDispatcher = dispatcher } - @PublishedApi - internal fun resetInjectedMain() { - injectedMainDispatcher = null - } + internal val IO: CoroutineDispatcher = DefaultIoScheduler } -internal expect fun createMainDispatcher(default: CoroutineDispatcher): MainCoroutineDispatcher +internal object DefaultIoScheduler : CoroutineDispatcher() { + // 2048 is an arbitrary KMP-friendly constant + private val unlimitedPool = newFixedThreadPoolContext(2048, "Dispatchers.IO") + private val io = unlimitedPool.limitedParallelism(64) // Default JVM size -private fun createDefaultDispatcherBasedOnMm(): CoroutineDispatcher { - return if (multithreadingSupported) createDefaultDispatcher() - else OldDefaultExecutor -} + @ExperimentalCoroutinesApi + override fun limitedParallelism(parallelism: Int): CoroutineDispatcher { + // See documentation to Dispatchers.IO for the rationale + return unlimitedPool.limitedParallelism(parallelism) + } -private fun takeEventLoop(): EventLoopImpl = - ThreadLocalEventLoop.currentOrNull() as? EventLoopImpl ?: - error("There is no event loop. Use runBlocking { ... } to start one.") - -internal object OldDefaultExecutor : CoroutineDispatcher(), Delay { - override fun dispatch(context: CoroutineContext, block: Runnable) = - takeEventLoop().dispatch(context, block) - override fun scheduleResumeAfterDelay(timeMillis: Long, continuation: CancellableContinuation<Unit>) = - takeEventLoop().scheduleResumeAfterDelay(timeMillis, continuation) - override fun invokeOnTimeout(timeMillis: Long, block: Runnable, context: CoroutineContext): DisposableHandle = - takeEventLoop().invokeOnTimeout(timeMillis, block, context) -} + override fun dispatch(context: CoroutineContext, block: Runnable) { + io.dispatch(context, block) + } + + @InternalCoroutinesApi + override fun dispatchYield(context: CoroutineContext, block: Runnable) { + io.dispatchYield(context, block) + } -internal class OldMainDispatcher(private val delegate: CoroutineDispatcher) : MainCoroutineDispatcher() { - override val immediate: MainCoroutineDispatcher - get() = throw UnsupportedOperationException("Immediate dispatching is not supported on Native") - override fun dispatch(context: CoroutineContext, block: Runnable) = delegate.dispatch(context, block) - override fun isDispatchNeeded(context: CoroutineContext): Boolean = delegate.isDispatchNeeded(context) - override fun dispatchYield(context: CoroutineContext, block: Runnable) = delegate.dispatchYield(context, block) - override fun toString(): String = toStringInternalImpl() ?: delegate.toString() + override fun toString(): String = "Dispatchers.IO" } + + +@Suppress("EXTENSION_SHADOWED_BY_MEMBER") +public actual val Dispatchers.IO: CoroutineDispatcher get() = IO + +internal expect fun createMainDispatcher(default: CoroutineDispatcher): MainCoroutineDispatcher diff --git a/kotlinx-coroutines-core/native/src/EventLoop.kt b/kotlinx-coroutines-core/native/src/EventLoop.kt index f4e5b8c9..25c3c12b 100644 --- a/kotlinx-coroutines-core/native/src/EventLoop.kt +++ b/kotlinx-coroutines-core/native/src/EventLoop.kt @@ -4,10 +4,6 @@ package kotlinx.coroutines -import kotlinx.cinterop.* -import kotlinx.coroutines.internal.* -import kotlinx.coroutines.internal.multithreadingSupported -import platform.posix.* import kotlin.coroutines.* import kotlin.native.concurrent.* import kotlin.system.* @@ -21,21 +17,13 @@ internal actual abstract class EventLoopImplPlatform : EventLoop() { } protected actual fun reschedule(now: Long, delayedTask: EventLoopImplBase.DelayedTask) { - if (multithreadingSupported) { - DefaultExecutor.invokeOnTimeout(now, delayedTask, EmptyCoroutineContext) - } else { - error("Cannot execute task because event loop was shut down") - } + DefaultExecutor.invokeOnTimeout(now, delayedTask, EmptyCoroutineContext) } } internal class EventLoopImpl: EventLoopImplBase() { - override fun invokeOnTimeout(timeMillis: Long, block: Runnable, context: CoroutineContext): DisposableHandle { - if (!multithreadingSupported) { - return scheduleInvokeOnTimeout(timeMillis, block) - } - return DefaultDelay.invokeOnTimeout(timeMillis, block, context) - } + override fun invokeOnTimeout(timeMillis: Long, block: Runnable, context: CoroutineContext): DisposableHandle = + DefaultDelay.invokeOnTimeout(timeMillis, block, context) } internal actual fun createEventLoop(): EventLoop = EventLoopImpl() diff --git a/kotlinx-coroutines-core/native/src/MultithreadedDispatchers.kt b/kotlinx-coroutines-core/native/src/MultithreadedDispatchers.kt index 1c1306c2..007d079a 100644 --- a/kotlinx-coroutines-core/native/src/MultithreadedDispatchers.kt +++ b/kotlinx-coroutines-core/native/src/MultithreadedDispatchers.kt @@ -4,23 +4,20 @@ package kotlinx.coroutines +import kotlinx.atomicfu.* import kotlinx.coroutines.channels.* import kotlinx.coroutines.internal.* import kotlin.coroutines.* import kotlin.native.concurrent.* - -@ExperimentalCoroutinesApi -public actual fun newSingleThreadContext(name: String): CloseableCoroutineDispatcher { - if (!multithreadingSupported) throw IllegalStateException("This API is only supported for experimental K/N memory model") - return WorkerDispatcher(name) -} +import kotlin.time.* +import kotlin.time.Duration.Companion.milliseconds public actual fun newFixedThreadPoolContext(nThreads: Int, name: String): CloseableCoroutineDispatcher { - if (!multithreadingSupported) throw IllegalStateException("This API is only supported for experimental K/N memory model") - require(nThreads >= 1) { "Expected at least one thread, but got: $nThreads"} + require(nThreads >= 1) { "Expected at least one thread, but got: $nThreads" } return MultiWorkerDispatcher(name, nThreads) } +@OptIn(ExperimentalTime::class) internal class WorkerDispatcher(name: String) : CloseableCoroutineDispatcher(), Delay { private val worker = Worker.start(name = name) @@ -29,12 +26,16 @@ internal class WorkerDispatcher(name: String) : CloseableCoroutineDispatcher(), } override fun scheduleResumeAfterDelay(timeMillis: Long, continuation: CancellableContinuation<Unit>) { - worker.executeAfter(timeMillis.toMicrosSafe()) { + val handle = schedule(timeMillis, Runnable { with(continuation) { resumeUndispatched(Unit) } - } + }) + continuation.disposeOnCancellation(handle) } - override fun invokeOnTimeout(timeMillis: Long, block: Runnable, context: CoroutineContext): DisposableHandle { + override fun invokeOnTimeout(timeMillis: Long, block: Runnable, context: CoroutineContext): DisposableHandle = + schedule(timeMillis, block) + + private fun schedule(timeMillis: Long, block: Runnable): DisposableHandle { // Workers don't have an API to cancel sent "executeAfter" block, but we are trying // to control the damage and reduce reachable objects by nulling out `block` // that may retain a lot of references, and leaving only an empty shell after a timely disposal @@ -49,45 +50,122 @@ internal class WorkerDispatcher(name: String) : CloseableCoroutineDispatcher(), override fun dispose() { disposableHolder.value = null } + + fun isDisposed() = disposableHolder.value == null + } + + fun Worker.runAfterDelay(block: DisposableBlock, targetMoment: TimeMark) { + if (block.isDisposed()) return + val durationUntilTarget = -targetMoment.elapsedNow() + val quantum = 100.milliseconds + if (durationUntilTarget > quantum) { + executeAfter(quantum.inWholeMicroseconds) { runAfterDelay(block, targetMoment) } + } else { + executeAfter(maxOf(0, durationUntilTarget.inWholeMicroseconds), block) + } } val disposableBlock = DisposableBlock(block) - worker.executeAfter(timeMillis.toMicrosSafe(), disposableBlock) + val targetMoment = TimeSource.Monotonic.markNow() + timeMillis.milliseconds + worker.runAfterDelay(disposableBlock, targetMoment) return disposableBlock } override fun close() { worker.requestTermination().result // Note: calling "result" blocks } - - private fun Long.toMicrosSafe(): Long { - val result = this * 1000 - return if (result > this) result else Long.MAX_VALUE - } } -private class MultiWorkerDispatcher(name: String, workersCount: Int) : CloseableCoroutineDispatcher() { +private class MultiWorkerDispatcher( + private val name: String, + workersCount: Int +) : CloseableCoroutineDispatcher() { private val tasksQueue = Channel<Runnable>(Channel.UNLIMITED) - private val workers = Array(workersCount) { Worker.start(name = "$name-$it") } - - init { - workers.forEach { w -> w.executeAfter(0L) { workerRunLoop() } } + private val availableWorkers = Channel<CancellableContinuation<Runnable>>(Channel.UNLIMITED) + private val workerPool = OnDemandAllocatingPool(workersCount) { + Worker.start(name = "$name-$it").apply { + executeAfter { workerRunLoop() } + } } + /** + * (number of tasks - number of workers) * 2 + (1 if closed) + */ + private val tasksAndWorkersCounter = atomic(0L) + + private inline fun Long.isClosed() = this and 1L == 1L + private inline fun Long.hasTasks() = this >= 2 + private inline fun Long.hasWorkers() = this < 0 + private fun workerRunLoop() = runBlocking { - for (task in tasksQueue) { - // TODO error handling - task.run() + while (true) { + val state = tasksAndWorkersCounter.getAndUpdate { + if (it.isClosed() && !it.hasTasks()) return@runBlocking + it - 2 + } + if (state.hasTasks()) { + // we promised to process a task, and there are some + tasksQueue.receive().run() + } else { + try { + suspendCancellableCoroutine { + val result = availableWorkers.trySend(it) + checkChannelResult(result) + }.run() + } catch (e: CancellationException) { + /** we are cancelled from [close] and thus will never get back to this branch of code, + but there may still be pending work, so we can't just exit here. */ + } + } } } + // a worker that promised to be here and should actually arrive, so we wait for it in a blocking manner. + private fun obtainWorker(): CancellableContinuation<Runnable> = + availableWorkers.tryReceive().getOrNull() ?: runBlocking { availableWorkers.receive() } + override fun dispatch(context: CoroutineContext, block: Runnable) { - // TODO handle rejections - tasksQueue.trySend(block) + val state = tasksAndWorkersCounter.getAndUpdate { + if (it.isClosed()) + throw IllegalStateException("Dispatcher $name was closed, attempted to schedule: $block") + it + 2 + } + if (state.hasWorkers()) { + // there are workers that have nothing to do, let's grab one of them + obtainWorker().resume(block) + } else { + workerPool.allocate() + // no workers are available, we must queue the task + val result = tasksQueue.trySend(block) + checkChannelResult(result) + } } override fun close() { - tasksQueue.close() - workers.forEach { it.requestTermination().result } + tasksAndWorkersCounter.getAndUpdate { if (it.isClosed()) it else it or 1L } + val workers = workerPool.close() // no new workers will be created + while (true) { + // check if there are workers that await tasks in their personal channels, we need to wake them up + val state = tasksAndWorkersCounter.getAndUpdate { + if (it.hasWorkers()) it + 2 else it + } + if (!state.hasWorkers()) + break + obtainWorker().cancel() + } + /* + * Here we cannot avoid waiting on `.result`, otherwise it will lead + * to a native memory leak, including a pthread handle. + */ + val requests = workers.map { it.requestTermination() } + requests.map { it.result } + } + + private fun checkChannelResult(result: ChannelResult<*>) { + if (!result.isSuccess) + throw IllegalStateException( + "Internal invariants of $this were violated, please file a bug to kotlinx.coroutines", + result.exceptionOrNull() + ) } } diff --git a/kotlinx-coroutines-core/native/src/internal/Concurrent.kt b/kotlinx-coroutines-core/native/src/internal/Concurrent.kt index f6e18dd5..17975e2e 100644 --- a/kotlinx-coroutines-core/native/src/internal/Concurrent.kt +++ b/kotlinx-coroutines-core/native/src/internal/Concurrent.kt @@ -5,7 +5,6 @@ package kotlinx.coroutines.internal import kotlinx.atomicfu.* -import kotlin.native.concurrent.* import kotlinx.atomicfu.locks.withLock as withLock2 @Suppress("ACTUAL_WITHOUT_EXPECT") @@ -13,8 +12,6 @@ internal actual typealias ReentrantLock = kotlinx.atomicfu.locks.SynchronizedObj internal actual inline fun <T> ReentrantLock.withLock(action: () -> T): T = this.withLock2(action) -internal actual fun <E> subscriberList(): MutableList<E> = CopyOnWriteList<E>() - internal actual fun <E> identitySet(expectedSize: Int): MutableSet<E> = HashSet() @@ -32,7 +29,3 @@ internal open class SuppressSupportingThrowableImpl : Throwable() { } } -// getter instead of a property due to the bug in the initialization dependencies tracking with '-Xir-property-lazy-initialization=disabled' that Ktor uses -@OptIn(ExperimentalStdlibApi::class) -internal val multithreadingSupported: Boolean - get() = kotlin.native.isExperimentalMM() diff --git a/kotlinx-coroutines-core/native/src/internal/CoroutineExceptionHandlerImpl.kt b/kotlinx-coroutines-core/native/src/internal/CoroutineExceptionHandlerImpl.kt new file mode 100644 index 00000000..43d776cb --- /dev/null +++ b/kotlinx-coroutines-core/native/src/internal/CoroutineExceptionHandlerImpl.kt @@ -0,0 +1,31 @@ +/* + * Copyright 2016-2022 JetBrains s.r.o. Use of this source code is governed by the Apache 2.0 license. + */ + +package kotlinx.coroutines.internal + +import kotlinx.coroutines.* +import kotlin.coroutines.* +import kotlin.native.* + +private val lock = SynchronizedObject() + +internal actual val platformExceptionHandlers: Collection<CoroutineExceptionHandler> + get() = synchronized(lock) { platformExceptionHandlers_ } + +private val platformExceptionHandlers_ = mutableSetOf<CoroutineExceptionHandler>() + +internal actual fun ensurePlatformExceptionHandlerLoaded(callback: CoroutineExceptionHandler) { + synchronized(lock) { + platformExceptionHandlers_ += callback + } +} + +@OptIn(ExperimentalStdlibApi::class) +internal actual fun propagateExceptionFinalResort(exception: Throwable) { + // log exception + processUnhandledException(exception) +} + +internal actual class DiagnosticCoroutineContextException actual constructor(context: CoroutineContext) : + RuntimeException(context.toString()) diff --git a/kotlinx-coroutines-core/native/src/internal/Synchronized.kt b/kotlinx-coroutines-core/native/src/internal/Synchronized.kt index edbd3fde..8a8ecfe3 100644 --- a/kotlinx-coroutines-core/native/src/internal/Synchronized.kt +++ b/kotlinx-coroutines-core/native/src/internal/Synchronized.kt @@ -17,4 +17,4 @@ public actual typealias SynchronizedObject = kotlinx.atomicfu.locks.Synchronized * @suppress **This an internal API and should not be used from general code.** */ @InternalCoroutinesApi -public actual inline fun <T> synchronized(lock: SynchronizedObject, block: () -> T): T = lock.withLock2(block) +public actual inline fun <T> synchronizedImpl(lock: SynchronizedObject, block: () -> T): T = lock.withLock2(block) diff --git a/kotlinx-coroutines-core/native/src/internal/ThreadLocal.kt b/kotlinx-coroutines-core/native/src/internal/ThreadLocal.kt index e1825d67..405cbfb6 100644 --- a/kotlinx-coroutines-core/native/src/internal/ThreadLocal.kt +++ b/kotlinx-coroutines-core/native/src/internal/ThreadLocal.kt @@ -4,9 +4,13 @@ package kotlinx.coroutines.internal -internal actual class CommonThreadLocal<T> actual constructor() { - private var value: T? = null +internal actual class CommonThreadLocal<T>(private val name: Symbol) { @Suppress("UNCHECKED_CAST") - actual fun get(): T = value as T - actual fun set(value: T) { this.value = value } + actual fun get(): T = Storage[name] as T + actual fun set(value: T) { Storage[name] = value } } + +internal actual fun <T> commonThreadLocal(name: Symbol): CommonThreadLocal<T> = CommonThreadLocal(name) + +@ThreadLocal +private object Storage: MutableMap<Symbol, Any?> by mutableMapOf() diff --git a/kotlinx-coroutines-core/native/test/ConcurrentTestUtilities.kt b/kotlinx-coroutines-core/native/test/ConcurrentTestUtilities.kt index 639b5fb1..6690972d 100644 --- a/kotlinx-coroutines-core/native/test/ConcurrentTestUtilities.kt +++ b/kotlinx-coroutines-core/native/test/ConcurrentTestUtilities.kt @@ -4,7 +4,6 @@ package kotlinx.coroutines.exceptions -import kotlinx.atomicfu.* import kotlinx.coroutines.internal.* import platform.posix.* import kotlin.native.concurrent.* @@ -31,7 +30,7 @@ internal actual typealias SuppressSupportingThrowable = SuppressSupportingThrowa actual val Throwable.suppressed: Array<Throwable> get() = (this as? SuppressSupportingThrowableImpl)?.suppressed ?: emptyArray() -@Suppress("EXTENSION_SHADOWED_BY_MEMBER") +@Suppress("EXTENSION_SHADOWED_BY_MEMBER", "unused") actual fun Throwable.printStackTrace() = printStackTrace() actual fun currentThreadName(): String = Worker.current.name diff --git a/kotlinx-coroutines-core/native/test/MultithreadedDispatchersTest.kt b/kotlinx-coroutines-core/native/test/MultithreadedDispatchersTest.kt new file mode 100644 index 00000000..501b7b5b --- /dev/null +++ b/kotlinx-coroutines-core/native/test/MultithreadedDispatchersTest.kt @@ -0,0 +1,83 @@ +/* + * Copyright 2016-2023 JetBrains s.r.o. Use of this source code is governed by the Apache 2.0 license. + */ + +package kotlinx.coroutines + +import kotlinx.atomicfu.* +import kotlinx.coroutines.channels.* +import kotlinx.coroutines.internal.* +import kotlin.native.concurrent.* +import kotlin.test.* +import kotlin.time.Duration.Companion.seconds + +private class BlockingBarrier(val n: Int) { + val counter = atomic(0) + val wakeUp = Channel<Unit>(n - 1) + fun await() { + val count = counter.addAndGet(1) + if (count == n) { + repeat(n - 1) { + runBlocking { + wakeUp.send(Unit) + } + } + } else if (count < n) { + runBlocking { + wakeUp.receive() + } + } + } +} + +class MultithreadedDispatchersTest { + /** + * Test that [newFixedThreadPoolContext] does not allocate more dispatchers than it needs to. + * Incidentally also tests that it will allocate enough workers for its needs. Otherwise, the test will hang. + */ + @Test + fun testNotAllocatingExtraDispatchers() { + val barrier = BlockingBarrier(2) + val lock = SynchronizedObject() + suspend fun spin(set: MutableSet<Worker>) { + repeat(100) { + synchronized(lock) { set.add(Worker.current) } + delay(1) + } + } + val dispatcher = newFixedThreadPoolContext(64, "test") + try { + runBlocking { + val encounteredWorkers = mutableSetOf<Worker>() + val coroutine1 = launch(dispatcher) { + barrier.await() + spin(encounteredWorkers) + } + val coroutine2 = launch(dispatcher) { + barrier.await() + spin(encounteredWorkers) + } + listOf(coroutine1, coroutine2).joinAll() + assertEquals(2, encounteredWorkers.size) + } + } finally { + dispatcher.close() + } + } + + /** + * Test that [newSingleThreadContext] will not wait for the cancelled scheduled coroutines before closing. + */ + @Test + fun timeoutsNotPreventingClosing(): Unit = runBlocking { + val dispatcher = WorkerDispatcher("test") + withContext(dispatcher) { + withTimeout(5.seconds) { + } + } + withTimeout(1.seconds) { + dispatcher.close() // should not wait for the timeout + yield() + } + } +} diff --git a/kotlinx-coroutines-core/native/test/TestBase.kt b/kotlinx-coroutines-core/native/test/TestBase.kt index 6fef4752..d7dfeeae 100644 --- a/kotlinx-coroutines-core/native/test/TestBase.kt +++ b/kotlinx-coroutines-core/native/test/TestBase.kt @@ -107,3 +107,5 @@ public actual open class TestBase actual constructor() { error("Too few unhandled exceptions $exCount, expected ${unhandled.size}") } } + +public actual val isJavaAndWindows: Boolean get() = false diff --git a/kotlinx-coroutines-core/native/test/TestBaseExtension.kt b/kotlinx-coroutines-core/native/test/TestBaseExtension.kt deleted file mode 100644 index fde2cde5..00000000 --- a/kotlinx-coroutines-core/native/test/TestBaseExtension.kt +++ /dev/null @@ -1,19 +0,0 @@ -/* - * Copyright 2016-2021 JetBrains s.r.o. Use of this source code is governed by the Apache 2.0 license. - */ - -/* - * Copyright 2016-2021 JetBrains s.r.o. Use of this source code is governed by the Apache 2.0 license. - */ -package kotlinx.coroutines - -import kotlinx.coroutines.internal.* - -actual fun TestBase.runMtTest( - expected: ((Throwable) -> Boolean)?, - unhandled: List<(Throwable) -> Boolean>, - block: suspend CoroutineScope.() -> Unit -) { - if (!multithreadingSupported) return - return runTest(expected, unhandled, block) -} diff --git a/kotlinx-coroutines-core/native/test/WorkerTest.kt b/kotlinx-coroutines-core/native/test/WorkerTest.kt index 8252ca65..7ae31b26 100644 --- a/kotlinx-coroutines-core/native/test/WorkerTest.kt +++ b/kotlinx-coroutines-core/native/test/WorkerTest.kt @@ -4,6 +4,7 @@ package kotlinx.coroutines +import kotlinx.coroutines.channels.* import kotlin.coroutines.* import kotlin.native.concurrent.* import kotlin.test.* @@ -34,4 +35,31 @@ class WorkerTest : TestBase() { }.result worker.requestTermination() } + + /** + * Test that [runBlocking] does not crash after [Worker.requestTermination] is called on the worker that runs it. + */ + @Test + fun testRunBlockingInTerminatedWorker() { + val workerInRunBlocking = Channel<Unit>() + val workerTerminated = Channel<Unit>() + val checkResumption = Channel<Unit>() + val finished = Channel<Unit>() + val worker = Worker.start() + worker.executeAfter(0) { + runBlocking { + workerInRunBlocking.send(Unit) + workerTerminated.receive() + checkResumption.receive() + finished.send(Unit) + } + } + runBlocking { + workerInRunBlocking.receive() + worker.requestTermination() + workerTerminated.send(Unit) + checkResumption.send(Unit) + finished.receive() + } + } } diff --git a/kotlinx-coroutines-core/native/test/internal/LinkedListTest.kt b/kotlinx-coroutines-core/native/test/internal/LinkedListTest.kt deleted file mode 100644 index 44ddf471..00000000 --- a/kotlinx-coroutines-core/native/test/internal/LinkedListTest.kt +++ /dev/null @@ -1,47 +0,0 @@ -/* - * Copyright 2016-2018 JetBrains s.r.o. Use of this source code is governed by the Apache 2.0 license. - */ - -package kotlinx.coroutines.internal - -import kotlin.test.Test -import kotlin.test.assertEquals -import kotlin.test.assertFalse -import kotlin.test.assertTrue - -class LinkedListTest { - data class IntNode(val i: Int) : LockFreeLinkedListNode() - - @Test - fun testSimpleAddLastRemove() { - val list = LockFreeLinkedListHead() - assertContents(list) - val n1 = IntNode(1).apply { list.addLast(this) } - assertContents(list, 1) - val n2 = IntNode(2).apply { list.addLast(this) } - assertContents(list, 1, 2) - val n3 = IntNode(3).apply { list.addLast(this) } - assertContents(list, 1, 2, 3) - val n4 = IntNode(4).apply { list.addLast(this) } - assertContents(list, 1, 2, 3, 4) - assertTrue(n1.remove()) - assertContents(list, 2, 3, 4) - assertTrue(n3.remove()) - assertContents(list, 2, 4) - assertTrue(n4.remove()) - assertContents(list, 2) - assertTrue(n2.remove()) - assertFalse(n2.remove()) - assertContents(list) - } - - private fun assertContents(list: LockFreeLinkedListHead, vararg expected: Int) { - val n = expected.size - val actual = IntArray(n) - var index = 0 - list.forEach<IntNode> { actual[index++] = it.i } - assertEquals(n, index) - for (i in 0 until n) assertEquals(expected[i], actual[i], "item i") - assertEquals(expected.isEmpty(), list.isEmpty) - } -} diff --git a/kotlinx-coroutines-core/nativeDarwin/src/Dispatchers.kt b/kotlinx-coroutines-core/nativeDarwin/src/Dispatchers.kt index ace20422..edc0a13c 100644 --- a/kotlinx-coroutines-core/nativeDarwin/src/Dispatchers.kt +++ b/kotlinx-coroutines-core/nativeDarwin/src/Dispatchers.kt @@ -5,7 +5,6 @@ package kotlinx.coroutines import kotlinx.cinterop.* -import kotlinx.coroutines.internal.* import platform.CoreFoundation.* import platform.darwin.* import kotlin.coroutines.* @@ -14,15 +13,14 @@ import kotlin.native.internal.NativePtr internal fun isMainThread(): Boolean = CFRunLoopGetCurrent() == CFRunLoopGetMain() -internal actual fun createMainDispatcher(default: CoroutineDispatcher): MainCoroutineDispatcher = - if (multithreadingSupported) DarwinMainDispatcher(false) else OldMainDispatcher(Dispatchers.Default) +internal actual fun createMainDispatcher(default: CoroutineDispatcher): MainCoroutineDispatcher = DarwinMainDispatcher(false) internal actual fun createDefaultDispatcher(): CoroutineDispatcher = DarwinGlobalQueueDispatcher private object DarwinGlobalQueueDispatcher : CoroutineDispatcher() { override fun dispatch(context: CoroutineContext, block: Runnable) { autoreleasepool { - dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT.convert(), 0)) { + dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT.convert(), 0u)) { block.run() } } diff --git a/kotlinx-coroutines-core/nativeDarwin/test/Launcher.kt b/kotlinx-coroutines-core/nativeDarwin/test/Launcher.kt index 78ed7659..3a2820e9 100644 --- a/kotlinx-coroutines-core/nativeDarwin/test/Launcher.kt +++ b/kotlinx-coroutines-core/nativeDarwin/test/Launcher.kt @@ -12,7 +12,7 @@ import kotlin.system.* // This is a separate entry point for tests in background fun mainBackground(args: Array<String>) { val worker = Worker.start(name = "main-background") - worker.execute(TransferMode.SAFE, { args.freeze() }) { + worker.execute(TransferMode.SAFE, { args }) { val result = testLauncherEntryPoint(it) exitProcess(result) } @@ -25,4 +25,4 @@ fun mainNoExit(args: Array<String>) { workerMain { // autoreleasepool to make sure interop objects are properly freed testLauncherEntryPoint(args) } -}
\ No newline at end of file +} diff --git a/kotlinx-coroutines-core/nativeDarwin/test/MainDispatcherTest.kt b/kotlinx-coroutines-core/nativeDarwin/test/MainDispatcherTest.kt index d460bd6e..9904f06c 100644 --- a/kotlinx-coroutines-core/nativeDarwin/test/MainDispatcherTest.kt +++ b/kotlinx-coroutines-core/nativeDarwin/test/MainDispatcherTest.kt @@ -4,7 +4,6 @@ package kotlinx.coroutines -import kotlinx.coroutines.internal.* import platform.CoreFoundation.* import platform.darwin.* import kotlin.coroutines.* @@ -13,7 +12,7 @@ import kotlin.test.* class MainDispatcherTest : TestBase() { private fun isMainThread(): Boolean = CFRunLoopGetCurrent() == CFRunLoopGetMain() - private fun canTestMainDispatcher() = !isMainThread() && multithreadingSupported + private fun canTestMainDispatcher() = !isMainThread() private fun runTestNotOnMainDispatcher(block: suspend CoroutineScope.() -> Unit) { // skip if already on the main thread, run blocking doesn't really work well with that diff --git a/kotlinx-coroutines-core/nativeOther/src/Dispatchers.kt b/kotlinx-coroutines-core/nativeOther/src/Dispatchers.kt index 517190d0..17278b0b 100644 --- a/kotlinx-coroutines-core/nativeOther/src/Dispatchers.kt +++ b/kotlinx-coroutines-core/nativeOther/src/Dispatchers.kt @@ -5,6 +5,7 @@ package kotlinx.coroutines import kotlin.coroutines.* +import kotlin.native.* internal actual fun createMainDispatcher(default: CoroutineDispatcher): MainCoroutineDispatcher = MissingMainDispatcher @@ -12,10 +13,9 @@ internal actual fun createMainDispatcher(default: CoroutineDispatcher): MainCoro internal actual fun createDefaultDispatcher(): CoroutineDispatcher = DefaultDispatcher private object DefaultDispatcher : CoroutineDispatcher() { - - // Delegated, so users won't be able to downcast and call 'close' - // The precise number of threads cannot be obtained until KT-48179 is implemented, 4 is just "good enough" number. - private val ctx = newFixedThreadPoolContext(4, "Dispatchers.Default") + // Be consistent with JVM -- at least 2 threads to provide some liveness guarantees in case of improper uses + @OptIn(ExperimentalStdlibApi::class) + private val ctx = newFixedThreadPoolContext(Platform.getAvailableProcessors().coerceAtLeast(2), "Dispatchers.Default") override fun dispatch(context: CoroutineContext, block: Runnable) { ctx.dispatch(context, block) diff --git a/kotlinx-coroutines-core/nativeOther/test/Launcher.kt b/kotlinx-coroutines-core/nativeOther/test/Launcher.kt index feddd4c0..58dbefcd 100644 --- a/kotlinx-coroutines-core/nativeOther/test/Launcher.kt +++ b/kotlinx-coroutines-core/nativeOther/test/Launcher.kt @@ -11,7 +11,7 @@ import kotlin.system.* // This is a separate entry point for tests in background fun mainBackground(args: Array<String>) { val worker = Worker.start(name = "main-background") - worker.execute(TransferMode.SAFE, { args.freeze() }) { + worker.execute(TransferMode.SAFE, { args }) { val result = testLauncherEntryPoint(it) exitProcess(result) }.result // block main thread @@ -20,4 +20,4 @@ fun mainBackground(args: Array<String>) { // This is a separate entry point for tests with leak checker fun mainNoExit(args: Array<String>) { testLauncherEntryPoint(args) -}
\ No newline at end of file +} diff --git a/kotlinx-coroutines-debug/README.md b/kotlinx-coroutines-debug/README.md index 5f302d20..104dde32 100644 --- a/kotlinx-coroutines-debug/README.md +++ b/kotlinx-coroutines-debug/README.md @@ -16,7 +16,7 @@ of coroutines hierarchy referenced by a [Job] or [CoroutineScope] instances usin This module also provides an automatic [BlockHound](https://github.com/reactor/BlockHound) integration that detects when a blocking operation was called in a coroutine context that prohibits it. In order to use it, please follow the BlockHound [quick start guide]( -https://github.com/reactor/BlockHound/blob/1.0.2.RELEASE/docs/quick_start.md). +https://github.com/reactor/BlockHound/blob/1.0.8.RELEASE/docs/quick_start.md). ### Using in your project @@ -61,7 +61,7 @@ stacktraces will be dumped to the console. ### Using as JVM agent Debug module can also be used as a standalone JVM agent to enable debug probes on the application startup. -You can run your application with an additional argument: `-javaagent:kotlinx-coroutines-debug-1.6.4.jar`. +You can run your application with an additional argument: `-javaagent:kotlinx-coroutines-debug-1.7.2.jar`. Additionally, on Linux and Mac OS X you can use `kill -5 $pid` command in order to force your application to print all alive coroutines. When used as Java agent, `"kotlinx.coroutines.debug.enable.creation.stack.trace"` system property can be used to control [DebugProbes.enableCreationStackTraces] along with agent startup. @@ -123,7 +123,7 @@ Coroutine "coroutine#2":DeferredCoroutine{Active}@289d1c02, state: SUSPENDED at ExampleKt.combineResults(Example.kt:11) at ExampleKt$computeValue$2.invokeSuspend(Example.kt:7) at ExampleKt$main$1$deferred$1.invokeSuspend(Example.kt:25) - (Coroutine creation stacktrace) + at _COROUTINE._CREATION._(CoroutineDebugging.kt) at kotlin.coroutines.intrinsics.IntrinsicsKt__IntrinsicsJvmKt.createCoroutineUnintercepted(IntrinsicsJvm.kt:116) at kotlinx.coroutines.intrinsics.CancellableKt.startCoroutineCancellable(Cancellable.kt:25) at kotlinx.coroutines.BuildersKt.async$default(Unknown Source) diff --git a/kotlinx-coroutines-debug/api/kotlinx-coroutines-debug.api b/kotlinx-coroutines-debug/api/kotlinx-coroutines-debug.api index 5bf70626..b671b1a4 100644 --- a/kotlinx-coroutines-debug/api/kotlinx-coroutines-debug.api +++ b/kotlinx-coroutines-debug/api/kotlinx-coroutines-debug.api @@ -18,6 +18,7 @@ public final class kotlinx/coroutines/debug/DebugProbes { public static synthetic fun dumpCoroutines$default (Lkotlinx/coroutines/debug/DebugProbes;Ljava/io/PrintStream;ILjava/lang/Object;)V public final fun dumpCoroutinesInfo ()Ljava/util/List; public final fun getEnableCreationStackTraces ()Z + public final fun getIgnoreCoroutinesWithEmptyContext ()Z public final fun getSanitizeStackTraces ()Z public final fun install ()V public final fun isInstalled ()Z @@ -28,6 +29,7 @@ public final class kotlinx/coroutines/debug/DebugProbes { public static synthetic fun printScope$default (Lkotlinx/coroutines/debug/DebugProbes;Lkotlinx/coroutines/CoroutineScope;Ljava/io/PrintStream;ILjava/lang/Object;)V public final fun scopeToString (Lkotlinx/coroutines/CoroutineScope;)Ljava/lang/String; public final fun setEnableCreationStackTraces (Z)V + public final fun setIgnoreCoroutinesWithEmptyContext (Z)V public final fun setSanitizeStackTraces (Z)V public final fun uninstall ()V public final fun withDebugProbes (Lkotlin/jvm/functions/Function0;)V diff --git a/kotlinx-coroutines-debug/build.gradle b/kotlinx-coroutines-debug/build.gradle index 9165a41f..42d0b8d0 100644 --- a/kotlinx-coroutines-debug/build.gradle +++ b/kotlinx-coroutines-debug/build.gradle @@ -4,6 +4,9 @@ apply plugin: "com.github.johnrengelman.shadow" +// apply plugin to use autocomplete for Kover DSL +apply plugin: 'org.jetbrains.kotlinx.kover' + configurations { shadowDeps // shaded dependencies, not included into the resulting .pom file compileOnly.extendsFrom(shadowDeps) @@ -30,20 +33,15 @@ java { disableAutoTargetJvm() } -jar { - setEnabled(false) +// This is required for BlockHound tests to work, see https://github.com/Kotlin/kotlinx.coroutines/issues/3701 +tasks.withType(Test).configureEach { + if (JavaVersion.current().isCompatibleWith(JavaVersion.VERSION_13)) { + jvmArgs += ["-XX:+AllowRedefinitionToAddDeleteMethods"] + } } -// This is a rough estimation of what shadow plugin has been doing with our default configuration prior to -// 1.6.2: https://github.com/johnrengelman/shadow/blob/1ff12fc816629ae5bc331fa3889c8ecfcaee7b27/src/main/groovy/com/github/jengelman/gradle/plugins/shadow/ShadowJavaPlugin.groovy#L72-L82 -// We just emulate it here for backwards compatibility -shadowJar.configure { - def classpath = project.objects.fileCollection().from { -> - project.configurations.findByName('runtimeClasspath') - } - doFirst { - manifest.attributes 'Class-Path': classpath.collect { "${it.name}" }.findAll { it }.join(' ') - } +jar { + setEnabled(false) } def shadowJarTask = shadowJar { @@ -65,14 +63,11 @@ configurations { } } -def commonKoverExcludes = - // Never used, safety mechanism - ["kotlinx.coroutines.debug.internal.NoOpProbesKt"] - -tasks.koverHtmlReport { - excludes = commonKoverExcludes -} - -tasks.koverVerify { - excludes = commonKoverExcludes +koverReport { + filters { + excludes { + // Never used, safety mechanism + classes("kotlinx.coroutines.debug.internal.NoOpProbesKt") + } + } } diff --git a/kotlinx-coroutines-debug/src/CoroutinesBlockHoundIntegration.kt b/kotlinx-coroutines-debug/src/CoroutinesBlockHoundIntegration.kt index a26c1928..99f2b190 100644 --- a/kotlinx-coroutines-debug/src/CoroutinesBlockHoundIntegration.kt +++ b/kotlinx-coroutines-debug/src/CoroutinesBlockHoundIntegration.kt @@ -1,5 +1,5 @@ /* - * Copyright 2016-2021 JetBrains s.r.o. Use of this source code is governed by the Apache 2.0 license. + * Copyright 2016-2023 JetBrains s.r.o. Use of this source code is governed by the Apache 2.0 license. */ @file:Suppress("INVISIBLE_REFERENCE", "INVISIBLE_MEMBER") @@ -10,6 +10,9 @@ import kotlinx.coroutines.scheduling.* import reactor.blockhound.* import reactor.blockhound.integration.* +/** + * @suppress + */ public class CoroutinesBlockHoundIntegration : BlockHoundIntegration { override fun applyTo(builder: BlockHound.Builder): Unit = with(builder) { @@ -106,43 +109,39 @@ public class CoroutinesBlockHoundIntegration : BlockHoundIntegration { } private fun BlockHound.Builder.allowBlockingCallsInChannels() { - allowBlockingCallsInArrayChannel() - allowBlockingCallsInBroadcastChannel() - allowBlockingCallsInConflatedChannel() + allowBlockingCallsInBroadcastChannels() + allowBlockingCallsInConflatedChannels() } /** - * Allows blocking inside [kotlinx.coroutines.channels.ArrayChannel]. + * Allows blocking inside [kotlinx.coroutines.channels.BroadcastChannel]. */ - private fun BlockHound.Builder.allowBlockingCallsInArrayChannel() { - for (method in listOf( - "pollInternal", "isEmpty", "isFull", "isClosedForReceive", "offerInternal", "offerSelectInternal", - "enqueueSend", "pollSelectInternal", "enqueueReceiveInternal", "onCancelIdempotent")) + private fun BlockHound.Builder.allowBlockingCallsInBroadcastChannels() { + for (method in listOf("openSubscription", "removeSubscriber", "send", "trySend", "registerSelectForSend", + "close", "cancelImpl", "isClosedForSend", "value", "valueOrNull")) { - allowBlockingCallsInside("kotlinx.coroutines.channels.ArrayChannel", method) + allowBlockingCallsInside("kotlinx.coroutines.channels.BroadcastChannelImpl", method) } - } - - /** - * Allows blocking inside [kotlinx.coroutines.channels.ArrayBroadcastChannel]. - */ - private fun BlockHound.Builder.allowBlockingCallsInBroadcastChannel() { - for (method in listOf("offerInternal", "offerSelectInternal", "updateHead")) { - allowBlockingCallsInside("kotlinx.coroutines.channels.ArrayBroadcastChannel", method) + for (method in listOf("cancelImpl")) { + allowBlockingCallsInside("kotlinx.coroutines.channels.BroadcastChannelImpl\$SubscriberConflated", method) } - for (method in listOf("checkOffer", "pollInternal", "pollSelectInternal")) { - allowBlockingCallsInside("kotlinx.coroutines.channels.ArrayBroadcastChannel\$Subscriber", method) + for (method in listOf("cancelImpl")) { + allowBlockingCallsInside("kotlinx.coroutines.channels.BroadcastChannelImpl\$SubscriberBuffered", method) } } /** - * Allows blocking inside [kotlinx.coroutines.channels.ConflatedChannel]. + * Allows blocking inside [kotlinx.coroutines.channels.ConflatedBufferedChannel]. */ - private fun BlockHound.Builder.allowBlockingCallsInConflatedChannel() { - for (method in listOf("offerInternal", "offerSelectInternal", "pollInternal", "pollSelectInternal", - "onCancelIdempotent", "isEmpty", "enqueueReceiveInternal")) + private fun BlockHound.Builder.allowBlockingCallsInConflatedChannels() { + for (method in listOf("receive", "receiveCatching", "tryReceive", "registerSelectForReceive", + "send", "trySend", "sendBroadcast", "registerSelectForSend", + "close", "cancelImpl", "isClosedForSend", "isClosedForReceive", "isEmpty")) { - allowBlockingCallsInside("kotlinx.coroutines.channels.ConflatedChannel", method) + allowBlockingCallsInside("kotlinx.coroutines.channels.ConflatedBufferedChannel", method) + } + for (method in listOf("hasNext")) { + allowBlockingCallsInside("kotlinx.coroutines.channels.ConflatedBufferedChannel\$ConflatedChannelIterator", method) } } diff --git a/kotlinx-coroutines-debug/src/DebugProbes.kt b/kotlinx-coroutines-debug/src/DebugProbes.kt index ed346d81..493b85e1 100644 --- a/kotlinx-coroutines-debug/src/DebugProbes.kt +++ b/kotlinx-coroutines-debug/src/DebugProbes.kt @@ -20,13 +20,20 @@ import kotlin.coroutines.* * asynchronous stack-traces and coroutine dumps (similar to [ThreadMXBean.dumpAllThreads] and `jstack` via [DebugProbes.dumpCoroutines]. * All introspecting methods throw [IllegalStateException] if debug probes were not installed. * - * Installed hooks: + * ### Consistency guarantees * + * All snapshotting operations (e.g. [dumpCoroutines]) are *weakly-consistent*, meaning that they happen + * concurrently with coroutines progressing their own state. These operations are guaranteed to observe + * each coroutine's state exactly once, but the state is not guaranteed to be the most recent before the operation. + * In practice, it means that for snapshotting operations in progress, for each concurrent coroutine either + * the state prior to the operation or the state that was reached during the current operation is observed. + * + * ### Installed hooks * * `probeCoroutineResumed` is invoked on every [Continuation.resume]. * * `probeCoroutineSuspended` is invoked on every continuation suspension. - * * `probeCoroutineCreated` is invoked on every coroutine creation using stdlib intrinsics. + * * `probeCoroutineCreated` is invoked on every coroutine creation. * - * Overhead: + * ### Overhead * * Every created coroutine is stored in a concurrent hash map and hash map is looked up and * updated on each suspension and resumption. * * If [DebugProbes.enableCreationStackTraces] is enabled, stack trace of the current thread is captured on @@ -39,6 +46,8 @@ public object DebugProbes { * Whether coroutine creation stack traces should be sanitized. * Sanitization removes all frames from `kotlinx.coroutines` package except * the first one and the last one to simplify diagnostic. + * + * `true` by default. */ public var sanitizeStackTraces: Boolean get() = DebugProbesImpl.sanitizeStackTraces @@ -52,6 +61,8 @@ public object DebugProbes { * thread is captured and attached to the coroutine. * This option can be useful during local debug sessions, but is recommended * to be disabled in production environments to avoid stack trace dumping overhead. + * + * `true` by default. */ public var enableCreationStackTraces: Boolean get() = DebugProbesImpl.enableCreationStackTraces @@ -60,6 +71,22 @@ public object DebugProbes { } /** + * Whether to ignore coroutines whose context is [EmptyCoroutineContext]. + * + * Coroutines with empty context are considered to be irrelevant for the concurrent coroutines' observability: + * - They do not contribute to any concurrent executions + * - They do not contribute to the (concurrent) system's liveness and/or deadlocks, as no other coroutines might wait for them + * - The typical usage of such coroutines is a combinator/builder/lookahead parser that can be debugged using more convenient tools. + * + * `true` by default. + */ + public var ignoreCoroutinesWithEmptyContext: Boolean + get() = DebugProbesImpl.ignoreCoroutinesWithEmptyContext + set(value) { + DebugProbesImpl.ignoreCoroutinesWithEmptyContext = value + } + + /** * Determines whether debug probes were [installed][DebugProbes.install]. */ public val isInstalled: Boolean get() = DebugProbesImpl.isInstalled @@ -115,13 +142,14 @@ public object DebugProbes { * Throws [IllegalStateException] if the scope has no a job in it. */ public fun printScope(scope: CoroutineScope, out: PrintStream = System.out): Unit = - printJob(scope.coroutineContext[Job] ?: error("Job is not present in the scope"), out) + printJob(scope.coroutineContext[Job] ?: error("Job is not present in the scope"), out) /** - * Returns all existing coroutines info. + * Returns all existing coroutines' info. * The resulting collection represents a consistent snapshot of all existing coroutines at the moment of invocation. */ - public fun dumpCoroutinesInfo(): List<CoroutineInfo> = DebugProbesImpl.dumpCoroutinesInfo().map { CoroutineInfo(it) } + public fun dumpCoroutinesInfo(): List<CoroutineInfo> = + DebugProbesImpl.dumpCoroutinesInfo().map { CoroutineInfo(it) } /** * Dumps all active coroutines into the given output stream, providing a consistent snapshot of all existing coroutines at the moment of invocation. @@ -134,7 +162,7 @@ public object DebugProbes { * * Coroutine "coroutine#42":StandaloneCoroutine{Active}@58fdd99, state: SUSPENDED * at MyClass$awaitData.invokeSuspend(MyClass.kt:37) - * (Coroutine creation stacktrace) + * at _COROUTINE._CREATION._(CoroutineDebugging.kt) * at MyClass.createIoRequest(MyClass.kt:142) * at MyClass.fetchData(MyClass.kt:154) * at MyClass.showData(MyClass.kt:31) diff --git a/kotlinx-coroutines-debug/src/module-info.java b/kotlinx-coroutines-debug/src/module-info.java new file mode 100644 index 00000000..2c7571ec --- /dev/null +++ b/kotlinx-coroutines-debug/src/module-info.java @@ -0,0 +1,14 @@ +module kotlinx.coroutines.debug { + requires java.management; + requires java.instrument; + requires kotlin.stdlib; + requires kotlinx.coroutines.core; + requires net.bytebuddy; + requires net.bytebuddy.agent; + requires org.junit.jupiter.api; + requires org.junit.platform.commons; + +// exports kotlinx.coroutines.debug; // already exported by kotlinx.coroutines.core + exports kotlinx.coroutines.debug.junit4; + exports kotlinx.coroutines.debug.junit5; +} diff --git a/kotlinx-coroutines-debug/test/BlockHoundTest.kt b/kotlinx-coroutines-debug/test/BlockHoundTest.kt index 3f588785..5ec767c3 100644 --- a/kotlinx-coroutines-debug/test/BlockHoundTest.kt +++ b/kotlinx-coroutines-debug/test/BlockHoundTest.kt @@ -1,4 +1,5 @@ package kotlinx.coroutines.debug + import kotlinx.coroutines.* import kotlinx.coroutines.channels.* import org.junit.* @@ -55,20 +56,22 @@ class BlockHoundTest : TestBase() { } @Test - fun testChannelNotBeingConsideredBlocking() = runTest { + fun testBroadcastChannelNotBeingConsideredBlocking() = runTest { withContext(Dispatchers.Default) { - // Copy of kotlinx.coroutines.channels.ArrayChannelTest.testSimple - val q = Channel<Int>(1) - check(q.isEmpty) - check(!q.isClosedForReceive) + // Copy of kotlinx.coroutines.channels.BufferedChannelTest.testSimple + val q = BroadcastChannel<Int>(1) + val s = q.openSubscription() check(!q.isClosedForSend) + check(s.isEmpty) + check(!s.isClosedForReceive) val sender = launch { q.send(1) q.send(2) } val receiver = launch { - q.receive() == 1 - q.receive() == 2 + s.receive() == 1 + s.receive() == 2 + s.cancel() } sender.join() receiver.join() @@ -76,7 +79,7 @@ class BlockHoundTest : TestBase() { } @Test - fun testConflatedChannelsNotBeingConsideredBlocking() = runTest { + fun testConflatedChannelNotBeingConsideredBlocking() = runTest { withContext(Dispatchers.Default) { val q = Channel<Int>(Channel.CONFLATED) check(q.isEmpty) @@ -110,5 +113,4 @@ class BlockHoundTest : TestBase() { } } } - } diff --git a/kotlinx-coroutines-debug/test/CoroutinesDumpTest.kt b/kotlinx-coroutines-debug/test/CoroutinesDumpTest.kt index fd027912..12573cf8 100644 --- a/kotlinx-coroutines-debug/test/CoroutinesDumpTest.kt +++ b/kotlinx-coroutines-debug/test/CoroutinesDumpTest.kt @@ -23,12 +23,13 @@ class CoroutinesDumpTest : DebugTestBase() { val found = DebugProbes.dumpCoroutinesInfo().single { it.job === deferred } verifyDump( "Coroutine \"coroutine#1\":DeferredCoroutine{Active}@1e4a7dd4, state: SUSPENDED\n" + - "\tat kotlinx.coroutines.debug.CoroutinesDumpTest.sleepingNestedMethod(CoroutinesDumpTest.kt:95)\n" + - "\tat kotlinx.coroutines.debug.CoroutinesDumpTest.sleepingOuterMethod(CoroutinesDumpTest.kt:88)\n" + - "\t(Coroutine creation stacktrace)\n" + - "\tat kotlin.coroutines.intrinsics.IntrinsicsKt__IntrinsicsJvmKt.createCoroutineUnintercepted(IntrinsicsJvm.kt:116)\n" + - "\tat kotlinx.coroutines.intrinsics.CancellableKt.startCoroutineCancellable(Cancellable.kt:23)\n" + - "\tat kotlinx.coroutines.CoroutineStart.invoke(CoroutineStart.kt:99)\n", + "\tat kotlinx.coroutines.debug.CoroutinesDumpTest.sleepingNestedMethod(CoroutinesDumpTest.kt)\n" + + "\tat kotlinx.coroutines.debug.CoroutinesDumpTest.sleepingOuterMethod(CoroutinesDumpTest.kt)\n" + + "\tat _COROUTINE._CREATION._(CoroutineDebugging.kt)\n" + + "\tat kotlin.coroutines.intrinsics.IntrinsicsKt__IntrinsicsJvmKt.createCoroutineUnintercepted(IntrinsicsJvm.kt)\n" + + "\tat kotlinx.coroutines.intrinsics.CancellableKt.startCoroutineCancellable(Cancellable.kt)\n" + + "\tat kotlinx.coroutines.intrinsics.CancellableKt.startCoroutineCancellable\$default(Cancellable.kt)\n" + + "\tat kotlinx.coroutines.CoroutineStart.invoke(CoroutineStart.kt)\n", ignoredCoroutine = "BlockingCoroutine" ) { deferred.cancel() @@ -48,19 +49,21 @@ class CoroutinesDumpTest : DebugTestBase() { verifyDump( "Coroutine \"coroutine#1\":DeferredCoroutine{Active}@227d9994, state: RUNNING\n" + "\tat java.lang.Thread.sleep(Native Method)\n" + - "\tat kotlinx.coroutines.debug.CoroutinesDumpTest.nestedActiveMethod(CoroutinesDumpTest.kt:141)\n" + - "\tat kotlinx.coroutines.debug.CoroutinesDumpTest.activeMethod(CoroutinesDumpTest.kt:133)\n" + - "\tat kotlinx.coroutines.debug.CoroutinesDumpTest\$testRunningCoroutine\$1$deferred\$1.invokeSuspend(CoroutinesDumpTest.kt:41)\n" + - "\t(Coroutine creation stacktrace)\n" + - "\tat kotlin.coroutines.intrinsics.IntrinsicsKt__IntrinsicsJvmKt.createCoroutineUnintercepted(IntrinsicsJvm.kt:116)\n" + - "\tat kotlinx.coroutines.intrinsics.CancellableKt.startCoroutineCancellable(Cancellable.kt:23)\n" + - "\tat kotlinx.coroutines.CoroutineStart.invoke(CoroutineStart.kt:99)\n" + - "\tat kotlinx.coroutines.AbstractCoroutine.start(AbstractCoroutine.kt:148)\n" + + "\tat kotlinx.coroutines.debug.CoroutinesDumpTest.nestedActiveMethod(CoroutinesDumpTest.kt)\n" + + "\tat kotlinx.coroutines.debug.CoroutinesDumpTest.activeMethod(CoroutinesDumpTest.kt)\n" + + "\tat kotlinx.coroutines.debug.CoroutinesDumpTest.access\$activeMethod(CoroutinesDumpTest.kt)\n" + + "\tat kotlinx.coroutines.debug.CoroutinesDumpTest\$testRunningCoroutine\$1\$deferred\$1.invokeSuspend(CoroutinesDumpTest.kt)\n" + + "\tat _COROUTINE._CREATION._(CoroutineDebugging.kt)\n" + + "\tat kotlin.coroutines.intrinsics.IntrinsicsKt__IntrinsicsJvmKt.createCoroutineUnintercepted(IntrinsicsJvm.kt)\n" + + "\tat kotlinx.coroutines.intrinsics.CancellableKt.startCoroutineCancellable(Cancellable.kt)\n" + + "\tat kotlinx.coroutines.intrinsics.CancellableKt.startCoroutineCancellable\$default(Cancellable.kt)\n" + + "\tat kotlinx.coroutines.CoroutineStart.invoke(CoroutineStart.kt)\n" + + "\tat kotlinx.coroutines.AbstractCoroutine.start(AbstractCoroutine.kt)\n" + "\tat kotlinx.coroutines.BuildersKt__Builders_commonKt.async(Builders.common.kt)\n" + "\tat kotlinx.coroutines.BuildersKt.async(Unknown Source)\n" + "\tat kotlinx.coroutines.BuildersKt__Builders_commonKt.async\$default(Builders.common.kt)\n" + "\tat kotlinx.coroutines.BuildersKt.async\$default(Unknown Source)\n" + - "\tat kotlinx.coroutines.debug.CoroutinesDumpTest.testRunningCoroutine(CoroutinesDumpTest.kt:49)", + "\tat kotlinx.coroutines.debug.CoroutinesDumpTest\$testRunningCoroutine\$1.invokeSuspend(CoroutinesDumpTest.kt)", ignoredCoroutine = "BlockingCoroutine" ) { deferred.cancel() @@ -79,18 +82,19 @@ class CoroutinesDumpTest : DebugTestBase() { verifyDump( "Coroutine \"coroutine#1\":DeferredCoroutine{Active}@1e4a7dd4, state: RUNNING\n" + "\tat java.lang.Thread.sleep(Native Method)\n" + - "\tat kotlinx.coroutines.debug.CoroutinesDumpTest.nestedActiveMethod(CoroutinesDumpTest.kt:111)\n" + - "\tat kotlinx.coroutines.debug.CoroutinesDumpTest.activeMethod(CoroutinesDumpTest.kt:106)\n" + - "\t(Coroutine creation stacktrace)\n" + - "\tat kotlin.coroutines.intrinsics.IntrinsicsKt__IntrinsicsJvmKt.createCoroutineUnintercepted(IntrinsicsJvm.kt:116)\n" + - "\tat kotlinx.coroutines.intrinsics.CancellableKt.startCoroutineCancellable(Cancellable.kt:23)\n" + - "\tat kotlinx.coroutines.CoroutineStart.invoke(CoroutineStart.kt:99)\n" + - "\tat kotlinx.coroutines.AbstractCoroutine.start(AbstractCoroutine.kt:148)\n" + + "\tat kotlinx.coroutines.debug.CoroutinesDumpTest.nestedActiveMethod(CoroutinesDumpTest.kt)\n" + + "\tat kotlinx.coroutines.debug.CoroutinesDumpTest.activeMethod(CoroutinesDumpTest.kt)\n" + + "\tat _COROUTINE._CREATION._(CoroutineDebugging.kt)\n" + + "\tat kotlin.coroutines.intrinsics.IntrinsicsKt__IntrinsicsJvmKt.createCoroutineUnintercepted(IntrinsicsJvm.kt)\n" + + "\tat kotlinx.coroutines.intrinsics.CancellableKt.startCoroutineCancellable(Cancellable.kt)\n" + + "\tat kotlinx.coroutines.intrinsics.CancellableKt.startCoroutineCancellable\$default(Cancellable.kt)\n" + + "\tat kotlinx.coroutines.CoroutineStart.invoke(CoroutineStart.kt)\n" + + "\tat kotlinx.coroutines.AbstractCoroutine.start(AbstractCoroutine.kt)\n" + "\tat kotlinx.coroutines.BuildersKt__Builders_commonKt.async(Builders.common.kt)\n" + "\tat kotlinx.coroutines.BuildersKt.async(Unknown Source)\n" + "\tat kotlinx.coroutines.BuildersKt__Builders_commonKt.async\$default(Builders.common.kt)\n" + "\tat kotlinx.coroutines.BuildersKt.async\$default(Unknown Source)\n" + - "\tat kotlinx.coroutines.debug.CoroutinesDumpTest.testRunningCoroutineWithSuspensionPoint(CoroutinesDumpTest.kt:71)", + "\tat kotlinx.coroutines.debug.CoroutinesDumpTest\$testRunningCoroutineWithSuspensionPoint\$1.invokeSuspend(CoroutinesDumpTest.kt)", ignoredCoroutine = "BlockingCoroutine" ) { deferred.cancel() @@ -126,9 +130,12 @@ class CoroutinesDumpTest : DebugTestBase() { "kotlinx.coroutines.BuildersKt.async\$default(Unknown Source)\n" + "kotlinx.coroutines.debug.CoroutinesDumpTest\$testCreationStackTrace\$1.invokeSuspend(CoroutinesDumpTest.kt)" if (!result.startsWith(expected)) { - println("=== Actual result") - println(result) - error("Does not start with expected lines") + error(""" + |Does not start with expected lines + |=== Actual result: + |$result + """.trimMargin() + ) } } diff --git a/kotlinx-coroutines-debug/test/DebugProbesTest.kt b/kotlinx-coroutines-debug/test/DebugProbesTest.kt index 4b394381..cbeeb311 100644 --- a/kotlinx-coroutines-debug/test/DebugProbesTest.kt +++ b/kotlinx-coroutines-debug/test/DebugProbesTest.kt @@ -6,6 +6,7 @@ package kotlinx.coroutines.debug import kotlinx.coroutines.* import org.junit.Test import java.util.concurrent.* +import java.util.concurrent.atomic.AtomicBoolean import kotlin.test.* class DebugProbesTest : DebugTestBase() { @@ -20,7 +21,7 @@ class DebugProbesTest : DebugTestBase() { val traces = listOf( "java.util.concurrent.ExecutionException\n" + "\tat kotlinx.coroutines.debug.DebugProbesTest\$createDeferred\$1.invokeSuspend(DebugProbesTest.kt:14)\n" + - "\t(Coroutine boundary)\n" + + "\tat _COROUTINE._BOUNDARY._(CoroutineDebugging.kt)\n" + "\tat kotlinx.coroutines.debug.DebugProbesTest.oneMoreNestedMethod(DebugProbesTest.kt:49)\n" + "\tat kotlinx.coroutines.debug.DebugProbesTest.nestedMethod(DebugProbesTest.kt:44)\n" + "\tat kotlinx.coroutines.debug.DebugProbesTest\$testAsync\$1.invokeSuspend(DebugProbesTest.kt:17)\n", @@ -40,11 +41,11 @@ class DebugProbesTest : DebugTestBase() { val traces = listOf( "java.util.concurrent.ExecutionException\n" + "\tat kotlinx.coroutines.debug.DebugProbesTest\$createDeferred\$1.invokeSuspend(DebugProbesTest.kt)\n" + - "\t(Coroutine boundary)\n" + + "\tat _COROUTINE._BOUNDARY._(CoroutineDebugging.kt)\n" + "\tat kotlinx.coroutines.debug.DebugProbesTest.oneMoreNestedMethod(DebugProbesTest.kt)\n" + "\tat kotlinx.coroutines.debug.DebugProbesTest.nestedMethod(DebugProbesTest.kt)\n" + "\tat kotlinx.coroutines.debug.DebugProbesTest\$testAsyncWithProbes\$1\$1.invokeSuspend(DebugProbesTest.kt:62)\n" + - "\t(Coroutine creation stacktrace)\n" + + "\tat _COROUTINE._CREATION._(CoroutineDebugging.kt)\n" + "\tat kotlin.coroutines.intrinsics.IntrinsicsKt__IntrinsicsJvmKt.createCoroutineUnintercepted(IntrinsicsJvm.kt)\n" + "\tat kotlinx.coroutines.intrinsics.CancellableKt.startCoroutineCancellable(Cancellable.kt)\n" + "\tat kotlinx.coroutines.intrinsics.CancellableKt.startCoroutineCancellable\$default(Cancellable.kt)\n" + @@ -71,11 +72,11 @@ class DebugProbesTest : DebugTestBase() { val traces = listOf( "java.util.concurrent.ExecutionException\n" + "\tat kotlinx.coroutines.debug.DebugProbesTest\$createDeferred\$1.invokeSuspend(DebugProbesTest.kt:16)\n" + - "\t(Coroutine boundary)\n" + + "\tat _COROUTINE._BOUNDARY._(CoroutineDebugging.kt)\n" + "\tat kotlinx.coroutines.debug.DebugProbesTest.oneMoreNestedMethod(DebugProbesTest.kt:71)\n" + "\tat kotlinx.coroutines.debug.DebugProbesTest.nestedMethod(DebugProbesTest.kt:66)\n" + "\tat kotlinx.coroutines.debug.DebugProbesTest\$testAsyncWithSanitizedProbes\$1\$1.invokeSuspend(DebugProbesTest.kt:87)\n" + - "\t(Coroutine creation stacktrace)\n" + + "\tat _COROUTINE._CREATION._(CoroutineDebugging.kt)\n" + "\tat kotlin.coroutines.intrinsics.IntrinsicsKt__IntrinsicsJvmKt.createCoroutineUnintercepted(IntrinsicsJvm.kt:116)\n" + "\tat kotlinx.coroutines.intrinsics.CancellableKt.startCoroutineCancellable(Cancellable.kt:23)\n" + "\tat kotlinx.coroutines.debug.DebugProbesTest.testAsyncWithSanitizedProbes(DebugProbesTest.kt:38)", @@ -100,4 +101,59 @@ class DebugProbesTest : DebugTestBase() { verifyStackTrace(e, traces) } } + + @Test + fun testMultipleConsecutiveProbeResumed() = runTest { + val job = launch { + expect(1) + foo() + expect(4) + delay(Long.MAX_VALUE) + expectUnreached() + } + yield() + yield() + expect(5) + val infos = DebugProbes.dumpCoroutinesInfo() + assertEquals(2, infos.size) + assertEquals(setOf(State.RUNNING, State.SUSPENDED), infos.map { it.state }.toSet()) + job.cancel() + finish(6) + } + + @Test + fun testMultipleConsecutiveProbeResumedAndLaterRunning() = runTest { + val reachedActiveStage = AtomicBoolean(false) + val job = launch(Dispatchers.Default) { + expect(1) + foo() + expect(4) + yield() + reachedActiveStage.set(true) + while (isActive) { + // Spin until test is done + } + } + while (!reachedActiveStage.get()) { + delay(10) + } + expect(5) + val infos = DebugProbes.dumpCoroutinesInfo() + assertEquals(2, infos.size) + assertEquals(setOf(State.RUNNING, State.RUNNING), infos.map { it.state }.toSet()) + job.cancel() + finish(6) + } + + private suspend fun foo() { + bar() + // Kill TCO + expect(3) + } + + + private suspend fun bar() { + yield() + expect(2) + } } diff --git a/kotlinx-coroutines-debug/test/DumpCoroutineInfoAsJsonAndReferencesTest.kt b/kotlinx-coroutines-debug/test/DumpCoroutineInfoAsJsonAndReferencesTest.kt index 4808470e..7d252196 100644 --- a/kotlinx-coroutines-debug/test/DumpCoroutineInfoAsJsonAndReferencesTest.kt +++ b/kotlinx-coroutines-debug/test/DumpCoroutineInfoAsJsonAndReferencesTest.kt @@ -30,6 +30,10 @@ class DumpCoroutineInfoAsJsonAndReferencesTest : DebugTestBase() { runTestWithNamedDeferred("Name") @Test + fun testDumpOfNamedCoroutineWithSpecialCharacters() = + runTestWithNamedDeferred("Name with\n \"special\" characters\\/\t\b") + + @Test fun testDumpWithNoCoroutines() { val dumpResult = DebugProbesImpl.dumpCoroutinesInfoAsJsonAndReferences() assertEquals(dumpResult.size, 4) diff --git a/kotlinx-coroutines-debug/test/RunningThreadStackMergeTest.kt b/kotlinx-coroutines-debug/test/RunningThreadStackMergeTest.kt index e7fdeede..89dd9148 100644 --- a/kotlinx-coroutines-debug/test/RunningThreadStackMergeTest.kt +++ b/kotlinx-coroutines-debug/test/RunningThreadStackMergeTest.kt @@ -1,7 +1,6 @@ /* * Copyright 2016-2020 JetBrains s.r.o. Use of this source code is governed by the Apache 2.0 license. */ -@file:Suppress("INVISIBLE_MEMBER", "INVISIBLE_REFERENCE") package kotlinx.coroutines.debug import kotlinx.coroutines.* @@ -20,7 +19,6 @@ class RunningThreadStackMergeTest : DebugTestBase() { launchCoroutine() awaitCoroutineStarted() verifyDump( - "Coroutine \"coroutine#1\":BlockingCoroutine{Active}@62230679", // <- this one is ignored "Coroutine \"coroutine#2\":StandaloneCoroutine{Active}@50284dc4, state: RUNNING\n" + "\tat jdk.internal.misc.Unsafe.park(Native Method)\n" + "\tat java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)\n" + @@ -32,9 +30,9 @@ class RunningThreadStackMergeTest : DebugTestBase() { "\tat kotlinx.coroutines.debug.RunningThreadStackMergeTest\$suspendingFunction\$2.invokeSuspend(RunningThreadStackMergeTest.kt:77)\n" + "\tat kotlinx.coroutines.debug.RunningThreadStackMergeTest.suspendingFunction(RunningThreadStackMergeTest.kt:75)\n" + "\tat kotlinx.coroutines.debug.RunningThreadStackMergeTest\$launchCoroutine\$1.invokeSuspend(RunningThreadStackMergeTest.kt:68)\n" + - "\t(Coroutine creation stacktrace)\n" + + "\tat _COROUTINE._CREATION._(CoroutineDebugging.kt)\n" + "\tat kotlin.coroutines.intrinsics.IntrinsicsKt__IntrinsicsJvmKt.createCoroutineUnintercepted(IntrinsicsJvm.kt:116)", - ignoredCoroutine = ":BlockingCoroutine" + ignoredCoroutine = "BlockingCoroutine" ) { coroutineBlocker.await() } @@ -75,7 +73,6 @@ class RunningThreadStackMergeTest : DebugTestBase() { awaitCoroutineStarted() Thread.sleep(10) verifyDump( - "Coroutine \"coroutine#1\":BlockingCoroutine{Active}@62230679", // <- this one is ignored "Coroutine \"coroutine#2\":StandaloneCoroutine{Active}@3aea3c67, state: RUNNING\n" + "\tat jdk.internal.misc.Unsafe.park(Native Method)\n" + "\tat java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)\n" + @@ -87,9 +84,9 @@ class RunningThreadStackMergeTest : DebugTestBase() { "\tat kotlinx.coroutines.debug.RunningThreadStackMergeTest\$suspendingFunctionWithContext\$2.invokeSuspend(RunningThreadStackMergeTest.kt:124)\n" + "\tat kotlinx.coroutines.debug.RunningThreadStackMergeTest.suspendingFunctionWithContext(RunningThreadStackMergeTest.kt:122)\n" + "\tat kotlinx.coroutines.debug.RunningThreadStackMergeTest\$launchEscapingCoroutine\$1.invokeSuspend(RunningThreadStackMergeTest.kt:116)\n" + - "\t(Coroutine creation stacktrace)\n" + + "\tat _COROUTINE._CREATION._(CoroutineDebugging.kt)\n" + "\tat kotlin.coroutines.intrinsics.IntrinsicsKt__IntrinsicsJvmKt.createCoroutineUnintercepted(IntrinsicsJvm.kt:116)", - ignoredCoroutine = ":BlockingCoroutine" + ignoredCoroutine = "BlockingCoroutine" ) { coroutineBlocker.await() } @@ -116,7 +113,6 @@ class RunningThreadStackMergeTest : DebugTestBase() { launchEscapingCoroutineWithoutContext() awaitCoroutineStarted() verifyDump( - "Coroutine \"coroutine#1\":BlockingCoroutine{Active}@62230679", // <- this one is ignored "Coroutine \"coroutine#2\":StandaloneCoroutine{Active}@3aea3c67, state: RUNNING\n" + "\tat jdk.internal.misc.Unsafe.park(Native Method)\n" + "\tat java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)\n" + @@ -126,9 +122,9 @@ class RunningThreadStackMergeTest : DebugTestBase() { "\tat kotlinx.coroutines.debug.RunningThreadStackMergeTest.nonSuspendingFun(RunningThreadStackMergeTest.kt:83)\n" + "\tat kotlinx.coroutines.debug.RunningThreadStackMergeTest.suspendingFunctionWithoutContext(RunningThreadStackMergeTest.kt:160)\n" + "\tat kotlinx.coroutines.debug.RunningThreadStackMergeTest\$launchEscapingCoroutineWithoutContext\$1.invokeSuspend(RunningThreadStackMergeTest.kt:153)\n" + - "\t(Coroutine creation stacktrace)\n" + + "\tat _COROUTINE._CREATION._(CoroutineDebugging.kt)\n" + "\tat kotlin.coroutines.intrinsics.IntrinsicsKt__IntrinsicsJvmKt.createCoroutineUnintercepted(IntrinsicsJvm.kt:116)", - ignoredCoroutine = ":BlockingCoroutine" + ignoredCoroutine = "BlockingCoroutine" ) { coroutineBlocker.await() } @@ -158,7 +154,7 @@ class RunningThreadStackMergeTest : DebugTestBase() { "\tat kotlinx.coroutines.debug.StacktraceUtilsKt.verifyDump(StacktraceUtils.kt)\n" + "\tat kotlinx.coroutines.debug.StacktraceUtilsKt.verifyDump\$default(StacktraceUtils.kt)\n" + "\tat kotlinx.coroutines.debug.RunningThreadStackMergeTest\$testRunBlocking\$1.invokeSuspend(RunningThreadStackMergeTest.kt)\n" + - "\t(Coroutine creation stacktrace)\n" + + "\tat _COROUTINE._CREATION._(CoroutineDebugging.kt)\n" + "\tat kotlin.coroutines.intrinsics.IntrinsicsKt__IntrinsicsJvmKt.createCoroutineUnintercepted(IntrinsicsJvm.kt)\n") } @@ -173,7 +169,8 @@ class RunningThreadStackMergeTest : DebugTestBase() { assertTrue(true) } - @Test + @Test // IDEA-specific debugger API test + @Suppress("INVISIBLE_MEMBER", "INVISIBLE_REFERENCE") fun testActiveThread() = runBlocking<Unit> { launchCoroutine() awaitCoroutineStarted() diff --git a/kotlinx-coroutines-debug/test/SanitizedProbesTest.kt b/kotlinx-coroutines-debug/test/SanitizedProbesTest.kt index fd1c2882..6afda168 100644 --- a/kotlinx-coroutines-debug/test/SanitizedProbesTest.kt +++ b/kotlinx-coroutines-debug/test/SanitizedProbesTest.kt @@ -9,7 +9,6 @@ import kotlinx.coroutines.* import kotlinx.coroutines.debug.* import kotlinx.coroutines.selects.* import org.junit.* -import org.junit.Ignore import org.junit.Test import java.util.concurrent.* import kotlin.test.* @@ -27,11 +26,11 @@ class SanitizedProbesTest : DebugTestBase() { val traces = listOf( "java.util.concurrent.ExecutionException\n" + "\tat definitely.not.kotlinx.coroutines.SanitizedProbesTest\$createDeferredNested\$1.invokeSuspend(SanitizedProbesTest.kt:97)\n" + - "\t(Coroutine boundary)\n" + + "\tat _COROUTINE._BOUNDARY._(CoroutineDebugging.kt)\n" + "\tat definitely.not.kotlinx.coroutines.SanitizedProbesTest.oneMoreNestedMethod(SanitizedProbesTest.kt:67)\n" + "\tat definitely.not.kotlinx.coroutines.SanitizedProbesTest.nestedMethod(SanitizedProbesTest.kt:61)\n" + "\tat definitely.not.kotlinx.coroutines.SanitizedProbesTest\$testRecoveredStackTrace\$1.invokeSuspend(SanitizedProbesTest.kt:50)\n" + - "\t(Coroutine creation stacktrace)\n" + + "\tat _COROUTINE._CREATION._(CoroutineDebugging.kt)\n" + "\tat kotlin.coroutines.intrinsics.IntrinsicsKt__IntrinsicsJvmKt.createCoroutineUnintercepted(IntrinsicsJvm.kt:116)\n" + "\tat kotlinx.coroutines.intrinsics.CancellableKt.startCoroutineCancellable(Cancellable.kt:23)\n" + "\tat kotlinx.coroutines.TestBase.runTest\$default(TestBase.kt:141)\n" + @@ -51,7 +50,7 @@ class SanitizedProbesTest : DebugTestBase() { verifyDump( "Coroutine \"coroutine#3\":BlockingCoroutine{Active}@7d68ef40, state: RUNNING\n" + "\tat java.lang.Thread.getStackTrace(Thread.java)\n" + - "\t(Coroutine creation stacktrace)\n" + + "\tat _COROUTINE._CREATION._(CoroutineDebugging.kt)\n" + "\tat kotlin.coroutines.intrinsics.IntrinsicsKt__IntrinsicsJvmKt.createCoroutineUnintercepted(IntrinsicsJvm.kt:116)\n" + "\tat kotlinx.coroutines.intrinsics.CancellableKt.startCoroutineCancellable(Cancellable.kt:23)\n" + "\tat kotlinx.coroutines.TestBase.runTest\$default(TestBase.kt:141)\n" + @@ -59,7 +58,7 @@ class SanitizedProbesTest : DebugTestBase() { "Coroutine \"coroutine#4\":DeferredCoroutine{Active}@75c072cb, state: SUSPENDED\n" + "\tat definitely.not.kotlinx.coroutines.SanitizedProbesTest\$createActiveDeferred\$1.invokeSuspend(SanitizedProbesTest.kt:63)\n" + - "\t(Coroutine creation stacktrace)\n" + + "\tat _COROUTINE._CREATION._(CoroutineDebugging.kt)\n" + "\tat kotlin.coroutines.intrinsics.IntrinsicsKt__IntrinsicsJvmKt.createCoroutineUnintercepted(IntrinsicsJvm.kt:116)\n" + "\tat kotlinx.coroutines.intrinsics.CancellableKt.startCoroutineCancellable(Cancellable.kt:23)\n" + "\tat kotlinx.coroutines.BuildersKt__Builders_commonKt.async\$default(Builders.common.kt)\n" + @@ -83,12 +82,12 @@ class SanitizedProbesTest : DebugTestBase() { expect(3) verifyDump("Coroutine \"coroutine#1\":BlockingCoroutine{Active}@35fc6dc4, state: RUNNING\n" + "\tat java.lang.Thread.getStackTrace(Thread.java:1552)\n" + // Skip the rest - "\t(Coroutine creation stacktrace)\n" + + "\tat _COROUTINE._CREATION._(CoroutineDebugging.kt)\n" + "\tat kotlin.coroutines.intrinsics.IntrinsicsKt__IntrinsicsJvmKt.createCoroutineUnintercepted(IntrinsicsJvm.kt:116)", "Coroutine \"coroutine#2\":StandaloneCoroutine{Active}@1b68b9a4, state: SUSPENDED\n" + "\tat definitely.not.kotlinx.coroutines.SanitizedProbesTest\$launchSelector\$1\$1\$1.invokeSuspend(SanitizedProbesTest.kt)\n" + - "\t(Coroutine creation stacktrace)\n" + + "\tat _COROUTINE._CREATION._(CoroutineDebugging.kt)\n" + "\tat kotlin.coroutines.intrinsics.IntrinsicsKt__IntrinsicsJvmKt.createCoroutineUnintercepted(IntrinsicsJvm.kt:116)\n" + "\tat kotlinx.coroutines.intrinsics.CancellableKt.startCoroutineCancellable(Cancellable.kt:25)\n" + "\tat kotlinx.coroutines.BuildersKt__Builders_commonKt.launch\$default(Builders.common.kt)\n" + diff --git a/kotlinx-coroutines-debug/test/ScopedBuildersTest.kt b/kotlinx-coroutines-debug/test/ScopedBuildersTest.kt index c7627255..801b74b1 100644 --- a/kotlinx-coroutines-debug/test/ScopedBuildersTest.kt +++ b/kotlinx-coroutines-debug/test/ScopedBuildersTest.kt @@ -17,7 +17,7 @@ class ScopedBuildersTest : DebugTestBase() { yield() verifyDump( "Coroutine \"coroutine#1\":BlockingCoroutine{Active}@16612a51, state: RUNNING\n" + - "\t(Coroutine creation stacktrace)\n" + + "\tat _COROUTINE._CREATION._(CoroutineDebugging.kt)\n" + "\tat kotlin.coroutines.intrinsics.IntrinsicsKt__IntrinsicsJvmKt.createCoroutineUnintercepted(IntrinsicsJvm.kt:116)\n", "Coroutine \"coroutine#2\":StandaloneCoroutine{Active}@6b53e23f, state: SUSPENDED\n" + @@ -25,7 +25,7 @@ class ScopedBuildersTest : DebugTestBase() { "\tat kotlinx.coroutines.debug.ScopedBuildersTest.doWithContext(ScopedBuildersTest.kt:47)\n" + "\tat kotlinx.coroutines.debug.ScopedBuildersTest\$doInScope\$2.invokeSuspend(ScopedBuildersTest.kt:41)\n" + "\tat kotlinx.coroutines.debug.ScopedBuildersTest\$testNestedScopes\$1\$job\$1.invokeSuspend(ScopedBuildersTest.kt:30)\n" + - "\t(Coroutine creation stacktrace)\n" + + "\tat _COROUTINE._CREATION._(CoroutineDebugging.kt)\n" + "\tat kotlin.coroutines.intrinsics.IntrinsicsKt__IntrinsicsJvmKt.createCoroutineUnintercepted(IntrinsicsJvm.kt:116)") job.cancelAndJoin() finish(4) diff --git a/kotlinx-coroutines-debug/test/StacktraceUtils.kt b/kotlinx-coroutines-debug/test/StacktraceUtils.kt index 9cc626f1..869e2975 100644 --- a/kotlinx-coroutines-debug/test/StacktraceUtils.kt +++ b/kotlinx-coroutines-debug/test/StacktraceUtils.kt @@ -9,27 +9,14 @@ import kotlin.test.* public fun String.trimStackTrace(): String = trimIndent() + // Remove source line .replace(Regex(":[0-9]+"), "") + // Remove coroutine id .replace(Regex("#[0-9]+"), "") + // Remove trace prefix: "java.base@11.0.16.1/java.lang.Thread.sleep" => "java.lang.Thread.sleep" .replace(Regex("(?<=\tat )[^\n]*/"), "") .replace(Regex("\t"), "") .replace("sun.misc.Unsafe.", "jdk.internal.misc.Unsafe.") // JDK8->JDK11 - .applyBackspace() - -public fun String.applyBackspace(): String { - val array = toCharArray() - val stack = CharArray(array.size) - var stackSize = -1 - for (c in array) { - if (c != '\b') { - stack[++stackSize] = c - } else { - --stackSize - } - } - - return String(stack, 0, stackSize + 1) -} public fun verifyStackTrace(e: Throwable, traces: List<String>) { val stacktrace = toStackTrace(e) @@ -74,7 +61,7 @@ public fun verifyDump(vararg traces: String, ignoredCoroutine: String? = null, f * `$$BlockHound$$_` prepended at the last component. */ private fun cleanBlockHoundTraces(frames: List<String>): List<String> { - var result = mutableListOf<String>() + val result = mutableListOf<String>() val blockHoundSubstr = "\$\$BlockHound\$\$_" var i = 0 while (i < frames.size) { @@ -87,39 +74,146 @@ private fun cleanBlockHoundTraces(frames: List<String>): List<String> { return result } -public fun verifyDump(vararg traces: String, ignoredCoroutine: String? = null) { - val baos = ByteArrayOutputStream() - DebugProbes.dumpCoroutines(PrintStream(baos)) - val wholeDump = baos.toString() - val trace = wholeDump.split("\n\n") - if (traces.isEmpty()) { - val filtered = trace.filter { ignoredCoroutine == null || !it.contains(ignoredCoroutine) } - assertEquals(1, filtered.count()) - assertTrue(filtered[0].startsWith("Coroutines dump")) - return +/** + * Removes all frames that contain "java.util.concurrent" in it. + * + * We do leverage Java's locks for proper rendezvous and to fix the coroutine stack's state, + * but this API doesn't have (nor expected to) stable stacktrace, so we are filtering all such + * frames out. + * + * See https://github.com/Kotlin/kotlinx.coroutines/issues/3700 for the example of failure + */ +private fun removeJavaUtilConcurrentTraces(frames: List<String>): List<String> = + frames.filter { !it.contains("java.util.concurrent") } + +private data class CoroutineDump( + val header: CoroutineDumpHeader, + val coroutineStackTrace: List<String>, + val threadStackTrace: List<String>, + val originDump: String, + val originHeader: String, +) { + companion object { + private val COROUTINE_CREATION_FRAME_REGEX = + "at _COROUTINE\\._CREATION\\._\\(.*\\)".toRegex() + + fun parse(dump: String, traceCleaner: ((List<String>) -> List<String>)? = null): CoroutineDump { + val lines = dump + .trimStackTrace() + .split("\n") + val header = CoroutineDumpHeader.parse(lines[0]) + val traceLines = lines.slice(1 until lines.size) + val cleanedTraceLines = if (traceCleaner != null) { + traceCleaner(traceLines) + } else { + traceLines + } + val coroutineStackTrace = mutableListOf<String>() + val threadStackTrace = mutableListOf<String>() + var trace = coroutineStackTrace + for (line in cleanedTraceLines) { + if (line.isEmpty()) { + continue + } + if (line.matches(COROUTINE_CREATION_FRAME_REGEX)) { + require(trace !== threadStackTrace) { + "Found more than one coroutine creation frame" + } + trace = threadStackTrace + continue + } + trace.add(line) + } + return CoroutineDump(header, coroutineStackTrace, threadStackTrace, dump, lines[0]) + } + } + + fun verify(expected: CoroutineDump) { + assertEquals( + expected.header, header, + "Coroutine stacktrace headers are not matched:\n\t- ${expected.originHeader}\n\t+ ${originHeader}\n" + ) + verifyStackTrace("coroutine stack", coroutineStackTrace, expected.coroutineStackTrace) + verifyStackTrace("thread stack", threadStackTrace, expected.threadStackTrace) } - // Drop "Coroutine dump" line - trace.withIndex().drop(1).forEach { (index, value) -> - if (ignoredCoroutine != null && value.contains(ignoredCoroutine)) { - return@forEach + + private fun verifyStackTrace(traceName: String, actualStackTrace: List<String>, expectedStackTrace: List<String>) { + // It is possible there are more stack frames in a dump than we check + for ((ix, expectedLine) in expectedStackTrace.withIndex()) { + val actualLine = actualStackTrace[ix] + assertEquals( + expectedLine, actualLine, + "Following lines from $traceName are not matched:\n\t- ${expectedLine}\n\t+ ${actualLine}\nActual dump:\n$originDump\n\n" + ) } + } +} - val expected = traces[index - 1].applyBackspace().split("\n\t(Coroutine creation stacktrace)\n", limit = 2) - val actual = value.applyBackspace().split("\n\t(Coroutine creation stacktrace)\n", limit = 2) - assertEquals(expected.size, actual.size, "Creation stacktrace should be part of the expected input. Whole dump:\n$wholeDump") - - expected.withIndex().forEach { (index, trace) -> - val actualTrace = actual[index].trimStackTrace().sanitizeAddresses() - val expectedTrace = trace.trimStackTrace().sanitizeAddresses() - val actualLines = cleanBlockHoundTraces(actualTrace.split("\n")) - val expectedLines = expectedTrace.split("\n") - for (i in expectedLines.indices) { - assertEquals(expectedLines[i], actualLines[i], "Whole dump:\n$wholeDump") +private data class CoroutineDumpHeader( + val name: String?, + val className: String, + val state: String, +) { + companion object { + /** + * Parses following strings: + * + * - Coroutine "coroutine#10":DeferredCoroutine{Active}@66d87651, state: RUNNING + * - Coroutine DeferredCoroutine{Active}@66d87651, state: RUNNING + * + * into: + * + * - `CoroutineDumpHeader(name = "coroutine", className = "DeferredCoroutine", state = "RUNNING")` + * - `CoroutineDumpHeader(name = null, className = "DeferredCoroutine", state = "RUNNING")` + */ + fun parse(header: String): CoroutineDumpHeader { + val (identFull, stateFull) = header.split(", ", limit = 2) + val nameAndClassName = identFull.removePrefix("Coroutine ").split('@', limit = 2)[0] + val (name, className) = nameAndClassName.split(':', limit = 2).let { parts -> + val (quotedName, classNameWithState) = if (parts.size == 1) { + null to parts[0] + } else { + parts[0] to parts[1] + } + val name = quotedName?.removeSurrounding("\"")?.split('#', limit = 2)?.get(0) + val className = classNameWithState.replace("\\{.*\\}".toRegex(), "") + name to className } + val state = stateFull.removePrefix("state: ") + return CoroutineDumpHeader(name, className, state) } } } +public fun verifyDump(vararg expectedTraces: String, ignoredCoroutine: String? = null) { + val baos = ByteArrayOutputStream() + DebugProbes.dumpCoroutines(PrintStream(baos)) + val wholeDump = baos.toString() + val traces = wholeDump.split("\n\n") + assertTrue(traces[0].startsWith("Coroutines dump")) + + val dumps = traces + // Drop "Coroutine dump" line + .drop(1) + // Parse dumps and filter out ignored coroutines + .mapNotNull { trace -> + val dump = CoroutineDump.parse(trace, { + removeJavaUtilConcurrentTraces(cleanBlockHoundTraces(it)) + }) + if (dump.header.className == ignoredCoroutine) { + null + } else { + dump + } + } + + assertEquals(expectedTraces.size, dumps.size) + dumps.zip(expectedTraces.map { CoroutineDump.parse(it, ::removeJavaUtilConcurrentTraces) }) + .forEach { (dump, expectedDump) -> + dump.verify(expectedDump) + } +} + public fun String.trimPackage() = replace("kotlinx.coroutines.debug.", "") public fun verifyPartialDump(createdCoroutinesCount: Int, vararg frames: String) { @@ -134,10 +228,3 @@ public fun verifyPartialDump(createdCoroutinesCount: Int, vararg frames: String) assertEquals(createdCoroutinesCount, DebugProbes.dumpCoroutinesInfo().size) assertTrue(matches) } - -private fun String.sanitizeAddresses(): String { - val index = indexOf("coroutine\"") - val next = indexOf(',', index) - if (index == -1 || next == -1) return this - return substring(0, index) + substring(next, length) -} diff --git a/kotlinx-coroutines-debug/test/StandardBuildersDebugTest.kt b/kotlinx-coroutines-debug/test/StandardBuildersDebugTest.kt new file mode 100644 index 00000000..7ca8f142 --- /dev/null +++ b/kotlinx-coroutines-debug/test/StandardBuildersDebugTest.kt @@ -0,0 +1,50 @@ +/* + * Copyright 2016-2023 JetBrains s.r.o. Use of this source code is governed by the Apache 2.0 license. + */ +package kotlinx.coroutines.debug + +import org.junit.Test +import kotlin.test.* + +class StandardBuildersDebugTest : DebugTestBase() { + + @Test + fun testBuildersAreMissingFromDumpByDefault() = runTest { + val (b1, b2) = createBuilders() + + val coroutines = DebugProbes.dumpCoroutinesInfo() + assertEquals(1, coroutines.size) + assertTrue { b1.hasNext() && b2.hasNext() } // Don't let GC collect our coroutines until the test is complete + } + + @Test + fun testBuildersCanBeEnabled() = runTest { + try { + DebugProbes.ignoreCoroutinesWithEmptyContext = false + val (b1, b2) = createBuilders() + val coroutines = DebugProbes.dumpCoroutinesInfo() + assertEquals(3, coroutines.size) + assertTrue { b1.hasNext() && b2.hasNext() } // Don't let GC collect our coroutines until the test is complete + } finally { + DebugProbes.ignoreCoroutinesWithEmptyContext = true + } + } + + private fun createBuilders(): Pair<Iterator<Int>, Iterator<Int>> { + val fromSequence = sequence { + while (true) { + yield(1) + } + }.iterator() + + val fromIterator = iterator { + while (true) { + yield(1) + } + } + // Start coroutines + fromIterator.hasNext() + fromSequence.hasNext() + return fromSequence to fromIterator + } +} diff --git a/kotlinx-coroutines-debug/test/junit4/CoroutinesTimeoutDisabledTracesTest.kt b/kotlinx-coroutines-debug/test/junit4/CoroutinesTimeoutDisabledTracesTest.kt index 2063090c..4352140b 100644 --- a/kotlinx-coroutines-debug/test/junit4/CoroutinesTimeoutDisabledTracesTest.kt +++ b/kotlinx-coroutines-debug/test/junit4/CoroutinesTimeoutDisabledTracesTest.kt @@ -22,7 +22,7 @@ class CoroutinesTimeoutDisabledTracesTest : TestBase(disableOutCheck = true) { "at kotlinx.coroutines.debug.junit4.CoroutinesTimeoutDisabledTracesTest.hangForever", "at kotlinx.coroutines.debug.junit4.CoroutinesTimeoutDisabledTracesTest.waitForHangJob" ), - notExpectedOutParts = listOf("Coroutine creation stacktrace"), + notExpectedOutParts = listOf("_COROUTINE._CREATION._"), error = TestTimedOutException::class.java ) ) diff --git a/kotlinx-coroutines-test/README.md b/kotlinx-coroutines-test/README.md index f45ccd0c..aa0606d5 100644 --- a/kotlinx-coroutines-test/README.md +++ b/kotlinx-coroutines-test/README.md @@ -26,7 +26,7 @@ Provided [TestDispatcher] implementations: Add `kotlinx-coroutines-test` to your project test dependencies: ``` dependencies { - testImplementation 'org.jetbrains.kotlinx:kotlinx-coroutines-test:1.6.4' + testImplementation 'org.jetbrains.kotlinx:kotlinx-coroutines-test:1.7.2' } ``` @@ -107,6 +107,8 @@ on Kotlin/JS. The main differences are the following: * **The calls to `delay` are automatically skipped**, preserving the relative execution order of the tasks. This way, it's possible to make tests finish more-or-less immediately. +* **The execution times out after 10 seconds**, cancelling the test coroutine to prevent tests from hanging forever + and eating up the CI resources. * **Controlling the virtual time**: in case just skipping delays is not sufficient, it's possible to more carefully guide the execution, advancing the virtual time by a duration, draining the queue of the awaiting tasks, or running the tasks scheduled at the present moment. @@ -115,6 +117,31 @@ on Kotlin/JS. The main differences are the following: Sometimes, especially when working with third-party code, it's impossible to mock all the dispatchers in use. [runTest] will handle the situations where some code runs in dispatchers not integrated with the test module. +## Timeout + +Test automatically time out after 10 seconds. For example, this test will fail with a timeout exception: + +```kotlin +@Test +fun testHanging() = runTest { + CompletableDeferred<Unit>().await() // will hang forever +} +``` + +In case the test is expected to take longer than 10 seconds, the timeout can be increased by passing the `timeout` +parameter: + +```kotlin +@Test +fun testTakingALongTime() = runTest(timeout = 30.seconds) { + val result = withContext(Dispatchers.Default) { + delay(20.seconds) // this delay is not in the test dispatcher and will not be skipped + 3 + } + assertEquals(3, result) +} +``` + ## Delay-skipping To test regular suspend functions, which may have a delay, just run them inside the [runTest] block. @@ -163,30 +190,35 @@ fun testWithMultipleDelays() = runTest { ## Controlling the virtual time -Inside [runTest], the following operations are supported: +Inside [runTest], the execution is scheduled by [TestCoroutineScheduler], which is a virtual time scheduler. +The scheduler has several special methods that allow controlling the virtual time: * `currentTime` gets the current virtual time. * `runCurrent()` runs the tasks that are scheduled at this point of virtual time. * `advanceUntilIdle()` runs all enqueued tasks until there are no more. * `advanceTimeBy(timeDelta)` runs the enqueued tasks until the current virtual time advances by `timeDelta`. +* `timeSource` returns a `TimeSource` that uses the virtual time. ```kotlin @Test fun testFoo() = runTest { launch { - println(1) // executes during runCurrent() - delay(1_000) // suspends until time is advanced by at least 1_000 - println(2) // executes during advanceTimeBy(2_000) - delay(500) // suspends until the time is advanced by another 500 ms - println(3) // also executes during advanceTimeBy(2_000) - delay(5_000) // will suspend by another 4_500 ms - println(4) // executes during advanceUntilIdle() + val workDuration = testScheduler.timeSource.measureTime { + println(1) // executes during runCurrent() + delay(1_000) // suspends until time is advanced by at least 1_000 + println(2) // executes during advanceTimeBy(2_000) + delay(500) // suspends until the time is advanced by another 500 ms + println(3) // also executes during advanceTimeBy(2_000) + delay(5_000) // will suspend by another 4_500 ms + println(4) // executes during advanceUntilIdle() + } + assertEquals(6500.milliseconds, workDuration) // the work took 6_500 ms of virtual time } // the child coroutine has not run yet - runCurrent() + testScheduler.runCurrent() // the child coroutine has called println(1), and is suspended on delay(1_000) - advanceTimeBy(2_000) // progress time, this will cause two calls to `delay` to resume + testScheduler.advanceTimeBy(2.seconds) // progress time, this will cause two calls to `delay` to resume // the child coroutine has called println(2) and println(3) and suspends for another 4_500 virtual milliseconds - advanceUntilIdle() // will run the child coroutine to completion + testScheduler.advanceUntilIdle() // will run the child coroutine to completion assertEquals(6500, currentTime) // the child coroutine finished at virtual time of 6_500 milliseconds } ``` @@ -265,6 +297,32 @@ fun testSubject() = scope.runTest { } ``` +## Running background work + +Sometimes, the fact that [runTest] waits for all the coroutines to finish is undesired. +For example, the system under test may need to receive data from coroutines that always run in the background. +Emulating such coroutines by launching them from the test body is not sufficient, because [runTest] will wait for them +to finish, which they never typically do. + +For these cases, there is a special coroutine scope: [TestScope.backgroundScope]. +Coroutines launched in it will be cancelled at the end of the test. + +```kotlin +@Test +fun testExampleBackgroundJob() = runTest { + val channel = Channel<Int>() + backgroundScope.launch { + var i = 0 + while (true) { + channel.send(i++) + } + } + repeat(100) { + assertEquals(it, channel.receive()) + } +} +``` + ## Eagerly entering `launch` and `async` blocks Some tests only test functionality and don't particularly care about the precise order in which coroutines are @@ -357,7 +415,7 @@ either dependency injection, a service locator, or a default parameter, if it is ### Status of the API -This API is experimental and it is may change before migrating out of experimental (while it is marked as +Many parts of the API is experimental, and it is may change before migrating out of experimental (while it is marked as [`@ExperimentalCoroutinesApi`][ExperimentalCoroutinesApi]). Changes during experimental may have deprecation applied when possible, but it is not advised to use the API in stable code before it leaves experimental due to possible breaking changes. @@ -388,6 +446,7 @@ If you have any suggestions for improvements to this experimental API please sha [setMain]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-test/kotlinx.coroutines.test/set-main.html [TestScope.testScheduler]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-test/kotlinx.coroutines.test/-test-scope/test-scheduler.html [TestScope.runTest]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-test/kotlinx.coroutines.test/run-test.html +[TestScope.backgroundScope]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-test/kotlinx.coroutines.test/-test-scope/background-scope.html [runCurrent]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-test/kotlinx.coroutines.test/run-current.html <!--- END --> diff --git a/kotlinx-coroutines-test/api/kotlinx-coroutines-test.api b/kotlinx-coroutines-test/api/kotlinx-coroutines-test.api index bf639235..64639f24 100644 --- a/kotlinx-coroutines-test/api/kotlinx-coroutines-test.api +++ b/kotlinx-coroutines-test/api/kotlinx-coroutines-test.api @@ -22,7 +22,11 @@ public final class kotlinx/coroutines/test/TestBuildersKt { public static final fun runTest (Lkotlinx/coroutines/test/TestScope;JLkotlin/jvm/functions/Function2;)V public static synthetic fun runTest$default (Lkotlin/coroutines/CoroutineContext;JLkotlin/jvm/functions/Function2;ILjava/lang/Object;)V public static synthetic fun runTest$default (Lkotlinx/coroutines/test/TestCoroutineScope;JLkotlin/jvm/functions/Function2;ILjava/lang/Object;)V - public static synthetic fun runTest$default (Lkotlinx/coroutines/test/TestScope;JLkotlin/jvm/functions/Function2;ILjava/lang/Object;)V + public static final synthetic fun runTest$default (Lkotlinx/coroutines/test/TestScope;JLkotlin/jvm/functions/Function2;ILjava/lang/Object;)V + public static final fun runTest-8Mi8wO0 (Lkotlin/coroutines/CoroutineContext;JLkotlin/jvm/functions/Function2;)V + public static final fun runTest-8Mi8wO0 (Lkotlinx/coroutines/test/TestScope;JLkotlin/jvm/functions/Function2;)V + public static synthetic fun runTest-8Mi8wO0$default (Lkotlin/coroutines/CoroutineContext;JLkotlin/jvm/functions/Function2;ILjava/lang/Object;)V + public static synthetic fun runTest-8Mi8wO0$default (Lkotlinx/coroutines/test/TestScope;JLkotlin/jvm/functions/Function2;ILjava/lang/Object;)V public static final fun runTestWithLegacyScope (Lkotlin/coroutines/CoroutineContext;JLkotlin/jvm/functions/Function2;)V public static synthetic fun runTestWithLegacyScope$default (Lkotlin/coroutines/CoroutineContext;JLkotlin/jvm/functions/Function2;ILjava/lang/Object;)V } @@ -63,9 +67,10 @@ public final class kotlinx/coroutines/test/TestCoroutineScheduler : kotlin/corou public static final field Key Lkotlinx/coroutines/test/TestCoroutineScheduler$Key; public fun <init> ()V public final fun advanceTimeBy (J)V + public final fun advanceTimeBy-LRDsOJo (J)V public final fun advanceUntilIdle ()V public final fun getCurrentTime ()J - public final fun getTimeSource ()Lkotlin/time/TimeSource; + public final fun getTimeSource ()Lkotlin/time/TimeSource$WithComparableMarks; public final fun runCurrent ()V } @@ -92,11 +97,12 @@ public final class kotlinx/coroutines/test/TestCoroutineScopeKt { public static final fun runCurrent (Lkotlinx/coroutines/test/TestCoroutineScope;)V } -public abstract class kotlinx/coroutines/test/TestDispatcher : kotlinx/coroutines/CoroutineDispatcher, kotlinx/coroutines/Delay { +public abstract class kotlinx/coroutines/test/TestDispatcher : kotlinx/coroutines/CoroutineDispatcher, kotlinx/coroutines/Delay, kotlinx/coroutines/DelayWithTimeoutDiagnostics { public fun delay (JLkotlin/coroutines/Continuation;)Ljava/lang/Object; public abstract fun getScheduler ()Lkotlinx/coroutines/test/TestCoroutineScheduler; public fun invokeOnTimeout (JLjava/lang/Runnable;Lkotlin/coroutines/CoroutineContext;)Lkotlinx/coroutines/DisposableHandle; public fun scheduleResumeAfterDelay (JLkotlinx/coroutines/CancellableContinuation;)V + public synthetic fun timeoutMessage-LRDsOJo (J)Ljava/lang/String; } public final class kotlinx/coroutines/test/TestDispatchers { @@ -113,10 +119,13 @@ public final class kotlinx/coroutines/test/TestScopeKt { public static final fun TestScope (Lkotlin/coroutines/CoroutineContext;)Lkotlinx/coroutines/test/TestScope; public static synthetic fun TestScope$default (Lkotlin/coroutines/CoroutineContext;ILjava/lang/Object;)Lkotlinx/coroutines/test/TestScope; public static final fun advanceTimeBy (Lkotlinx/coroutines/test/TestScope;J)V + public static final fun advanceTimeBy-HG0u8IE (Lkotlinx/coroutines/test/TestScope;J)V public static final fun advanceUntilIdle (Lkotlinx/coroutines/test/TestScope;)V + public static final fun getCatchNonTestRelatedExceptions ()Z public static final fun getCurrentTime (Lkotlinx/coroutines/test/TestScope;)J - public static final fun getTestTimeSource (Lkotlinx/coroutines/test/TestScope;)Lkotlin/time/TimeSource; + public static final fun getTestTimeSource (Lkotlinx/coroutines/test/TestScope;)Lkotlin/time/TimeSource$WithComparableMarks; public static final fun runCurrent (Lkotlinx/coroutines/test/TestScope;)V + public static final fun setCatchNonTestRelatedExceptions (Z)V } public abstract interface class kotlinx/coroutines/test/UncaughtExceptionCaptor { diff --git a/kotlinx-coroutines-test/build.gradle.kts b/kotlinx-coroutines-test/build.gradle.kts index 7b244bb0..c968fc49 100644 --- a/kotlinx-coroutines-test/build.gradle.kts +++ b/kotlinx-coroutines-test/build.gradle.kts @@ -2,6 +2,8 @@ * Copyright 2016-2021 JetBrains s.r.o. Use of this source code is governed by the Apache 2.0 license. */ +import org.jetbrains.kotlin.gradle.plugin.mpp.* + val experimentalAnnotations = listOf( "kotlin.Experimental", "kotlinx.coroutines.ExperimentalCoroutinesApi", @@ -10,4 +12,19 @@ val experimentalAnnotations = listOf( kotlin { sourceSets.all { configureMultiplatform() } + + targets.withType(KotlinNativeTargetWithTests::class.java).configureEach { + binaries.getTest("DEBUG").apply { + optimized = true + binaryOptions["memoryModel"] = "experimental" + } + } + + sourceSets { + jvmTest { + dependencies { + implementation(project(":kotlinx-coroutines-debug")) + } + } + } } diff --git a/kotlinx-coroutines-test/common/src/TestBuilders.kt b/kotlinx-coroutines-test/common/src/TestBuilders.kt index 80dc8d60..f95dabc3 100644 --- a/kotlinx-coroutines-test/common/src/TestBuilders.kt +++ b/kotlinx-coroutines-test/common/src/TestBuilders.kt @@ -7,9 +7,14 @@ package kotlinx.coroutines.test import kotlinx.coroutines.* +import kotlinx.coroutines.flow.* import kotlinx.coroutines.selects.* import kotlin.coroutines.* import kotlin.jvm.* +import kotlin.time.* +import kotlin.time.Duration.Companion.milliseconds +import kotlin.time.Duration.Companion.seconds +import kotlinx.coroutines.internal.* /** * A test result. @@ -27,13 +32,144 @@ import kotlin.jvm.* * * Don't nest functions returning a [TestResult]. */ @Suppress("NO_ACTUAL_FOR_EXPECT") -@ExperimentalCoroutinesApi public expect class TestResult /** * Executes [testBody] as a test in a new coroutine, returning [TestResult]. * * On JVM and Native, this function behaves similarly to `runBlocking`, with the difference that the code that it runs + * will skip delays. This allows to use [delay] in tests without causing them to take more time than necessary. + * On JS, this function creates a `Promise` that executes the test body with the delay-skipping behavior. + * + * ``` + * @Test + * fun exampleTest() = runTest { + * val deferred = async { + * delay(1.seconds) + * async { + * delay(1.seconds) + * }.await() + * } + * + * deferred.await() // result available immediately + * } + * ``` + * + * The platform difference entails that, in order to use this function correctly in common code, one must always + * immediately return the produced [TestResult] from the test method, without doing anything else afterwards. See + * [TestResult] for details on this. + * + * The test is run on a single thread, unless other [CoroutineDispatcher] are used for child coroutines. + * Because of this, child coroutines are not executed in parallel to the test body. + * In order for the spawned-off asynchronous code to actually be executed, one must either [yield] or suspend the + * test body some other way, or use commands that control scheduling (see [TestCoroutineScheduler]). + * + * ``` + * @Test + * fun exampleWaitingForAsyncTasks1() = runTest { + * // 1 + * val job = launch { + * // 3 + * } + * // 2 + * job.join() // the main test coroutine suspends here, so the child is executed + * // 4 + * } + * + * @Test + * fun exampleWaitingForAsyncTasks2() = runTest { + * // 1 + * launch { + * // 3 + * } + * // 2 + * testScheduler.advanceUntilIdle() // runs the tasks until their queue is empty + * // 4 + * } + * ``` + * + * ### Task scheduling + * + * Delay skipping is achieved by using virtual time. + * If [Dispatchers.Main] is set to a [TestDispatcher] via [Dispatchers.setMain] before the test, + * then its [TestCoroutineScheduler] is used; + * otherwise, a new one is automatically created (or taken from [context] in some way) and can be used to control + * the virtual time, advancing it, running the tasks scheduled at a specific time etc. + * The scheduler can be accessed via [TestScope.testScheduler]. + * + * Delays in code that runs inside dispatchers that don't use a [TestCoroutineScheduler] don't get skipped: + * ``` + * @Test + * fun exampleTest() = runTest { + * val elapsed = TimeSource.Monotonic.measureTime { + * val deferred = async { + * delay(1.seconds) // will be skipped + * withContext(Dispatchers.Default) { + * delay(5.seconds) // Dispatchers.Default doesn't know about TestCoroutineScheduler + * } + * } + * deferred.await() + * } + * println(elapsed) // about five seconds + * } + * ``` + * + * ### Failures + * + * #### Test body failures + * + * If the created coroutine completes with an exception, then this exception will be thrown at the end of the test. + * + * #### Timing out + * + * There's a built-in timeout of 10 seconds for the test body. If the test body doesn't complete within this time, + * then the test fails with an [AssertionError]. The timeout can be changed by setting the [timeout] parameter. + * + * On timeout, the test body is cancelled so that the test finishes. If the code inside the test body does not + * respond to cancellation, the timeout will not be able to make the test execution stop. + * In that case, the test will hang despite the attempt to terminate it. + * + * On the JVM, if `DebugProbes` from the `kotlinx-coroutines-debug` module are installed, the current dump of the + * coroutines' stack is printed to the console on timeout before the test body is cancelled. + * + * #### Reported exceptions + * + * Unhandled exceptions will be thrown at the end of the test. + * If uncaught exceptions happen after the test finishes, they are propagated in a platform-specific manner: + * see [handleCoroutineException] for details. + * If the test coroutine completes with an exception, the unhandled exceptions are suppressed by it. + * + * #### Uncompleted coroutines + * + * Otherwise, the test will hang until all the coroutines launched inside [testBody] complete. + * This may be an issue when there are some coroutines that are not supposed to complete, like infinite loops that + * perform some background work and are supposed to outlive the test. + * In that case, [TestScope.backgroundScope] can be used to launch such coroutines. + * They will be cancelled automatically when the test finishes. + * + * ### Configuration + * + * [context] can be used to affect the environment of the code under test. Beside just being passed to the coroutine + * scope created for the test, [context] also can be used to change how the test is executed. + * See the [TestScope] constructor function documentation for details. + * + * @throws IllegalArgumentException if the [context] is invalid. See the [TestScope] constructor docs for details. + */ +public fun runTest( + context: CoroutineContext = EmptyCoroutineContext, + timeout: Duration = DEFAULT_TIMEOUT, + testBody: suspend TestScope.() -> Unit +): TestResult { + check(context[RunningInRunTest] == null) { + "Calls to `runTest` can't be nested. Please read the docs on `TestResult` for details." + } + return TestScope(context + RunningInRunTest).runTest(timeout, testBody) +} + +/** + * Executes [testBody] as a test in a new coroutine, returning [TestResult]. + * + * On JVM and Native, this function behaves similarly to `runBlocking`, with the difference that the code that it runs * will skip delays. This allows to use [delay] in without causing the tests to take more time than necessary. * On JS, this function creates a `Promise` that executes the test body with the delay-skipping behavior. * @@ -41,9 +177,9 @@ public expect class TestResult * @Test * fun exampleTest() = runTest { * val deferred = async { - * delay(1_000) + * delay(1.seconds) * async { - * delay(1_000) + * delay(1.seconds) * }.await() * } * @@ -57,7 +193,7 @@ public expect class TestResult * * The test is run in a single thread, unless other [CoroutineDispatcher] are used for child coroutines. * Because of this, child coroutines are not executed in parallel to the test body. - * In order to for the spawned-off asynchronous code to actually be executed, one must either [yield] or suspend the + * In order for the spawned-off asynchronous code to actually be executed, one must either [yield] or suspend the * test body some other way, or use commands that control scheduling (see [TestCoroutineScheduler]). * * ``` @@ -99,9 +235,9 @@ public expect class TestResult * fun exampleTest() = runTest { * val elapsed = TimeSource.Monotonic.measureTime { * val deferred = async { - * delay(1_000) // will be skipped + * delay(1.seconds) // will be skipped * withContext(Dispatchers.Default) { - * delay(5_000) // Dispatchers.Default doesn't know about TestCoroutineScheduler + * delay(5.seconds) // Dispatchers.Default doesn't know about TestCoroutineScheduler * } * } * deferred.await() @@ -131,7 +267,7 @@ public expect class TestResult * * In the general case, if there are active jobs, it's impossible to detect if they are going to complete eventually due * to the asynchronous nature of coroutines. In order to prevent tests hanging in this scenario, [runTest] will wait - * for [dispatchTimeoutMs] milliseconds (by default, 60 seconds) from the moment when [TestCoroutineScheduler] becomes + * for [dispatchTimeoutMs] from the moment when [TestCoroutineScheduler] becomes * idle before throwing [AssertionError]. If some dispatcher linked to [TestCoroutineScheduler] receives a * task during that time, the timer gets reset. * @@ -143,31 +279,124 @@ public expect class TestResult * * @throws IllegalArgumentException if the [context] is invalid. See the [TestScope] constructor docs for details. */ -@ExperimentalCoroutinesApi +@Deprecated( + "Define a total timeout for the whole test instead of using dispatchTimeoutMs. " + + "Warning: the proposed replacement is not identical as it uses 'dispatchTimeoutMs' as the timeout for the whole test!", + ReplaceWith("runTest(context, timeout = dispatchTimeoutMs.milliseconds, testBody)", + "kotlin.time.Duration.Companion.milliseconds"), + DeprecationLevel.WARNING +) // Warning since 1.7.0, was experimental in 1.6.x public fun runTest( context: CoroutineContext = EmptyCoroutineContext, - dispatchTimeoutMs: Long = DEFAULT_DISPATCH_TIMEOUT_MS, + dispatchTimeoutMs: Long, testBody: suspend TestScope.() -> Unit ): TestResult { if (context[RunningInRunTest] != null) throw IllegalStateException("Calls to `runTest` can't be nested. Please read the docs on `TestResult` for details.") - return TestScope(context + RunningInRunTest).runTest(dispatchTimeoutMs, testBody) + @Suppress("DEPRECATION") + return TestScope(context + RunningInRunTest).runTest(dispatchTimeoutMs = dispatchTimeoutMs, testBody) +} + +/** + * Performs [runTest] on an existing [TestScope]. See the documentation for [runTest] for details. + */ +public fun TestScope.runTest( + timeout: Duration = DEFAULT_TIMEOUT, + testBody: suspend TestScope.() -> Unit +): TestResult = asSpecificImplementation().let { scope -> + scope.enter() + createTestResult { + /** TODO: moving this [AbstractCoroutine.start] call outside [createTestResult] fails on JS. */ + scope.start(CoroutineStart.UNDISPATCHED, scope) { + /* we're using `UNDISPATCHED` to avoid the event loop, but we do want to set up the timeout machinery + before any code executes, so we have to park here. */ + yield() + testBody() + } + var timeoutError: Throwable? = null + var cancellationException: CancellationException? = null + val workRunner = launch(CoroutineName("kotlinx.coroutines.test runner")) { + while (true) { + val executedSomething = testScheduler.tryRunNextTaskUnless { !isActive } + if (executedSomething) { + /** yield to check for cancellation. On JS, we can't use [ensureActive] here, as the cancellation + * procedure needs a chance to run concurrently. */ + yield() + } else { + // waiting for the next task to be scheduled, or for the test runner to be cancelled + testScheduler.receiveDispatchEvent() + } + } + } + try { + withTimeout(timeout) { + coroutineContext.job.invokeOnCompletion(onCancelling = true) { exception -> + if (exception is TimeoutCancellationException) { + dumpCoroutines() + val activeChildren = scope.children.filter(Job::isActive).toList() + val completionCause = if (scope.isCancelled) scope.tryGetCompletionCause() else null + var message = "After waiting for $timeout" + if (completionCause == null) + message += ", the test coroutine is not completing" + if (activeChildren.isNotEmpty()) + message += ", there were active child jobs: $activeChildren" + if (completionCause != null && activeChildren.isEmpty()) { + message += if (scope.isCompleted) + ", the test coroutine completed" + else + ", the test coroutine was not completed" + } + timeoutError = UncompletedCoroutinesError(message) + cancellationException = CancellationException("The test timed out") + (scope as Job).cancel(cancellationException!!) + } + } + scope.join() + workRunner.cancelAndJoin() + } + } catch (_: TimeoutCancellationException) { + scope.join() + val completion = scope.getCompletionExceptionOrNull() + if (completion != null && completion !== cancellationException) { + timeoutError!!.addSuppressed(completion) + } + workRunner.cancelAndJoin() + } finally { + backgroundScope.cancel() + testScheduler.advanceUntilIdleOr { false } + val uncaughtExceptions = scope.leave() + throwAll(timeoutError ?: scope.getCompletionExceptionOrNull(), uncaughtExceptions) + } + } } /** * Performs [runTest] on an existing [TestScope]. + * + * In the general case, if there are active jobs, it's impossible to detect if they are going to complete eventually due + * to the asynchronous nature of coroutines. In order to prevent tests hanging in this scenario, [runTest] will wait + * for [dispatchTimeoutMs] from the moment when [TestCoroutineScheduler] becomes + * idle before throwing [AssertionError]. If some dispatcher linked to [TestCoroutineScheduler] receives a + * task during that time, the timer gets reset. */ -@ExperimentalCoroutinesApi +@Deprecated( + "Define a total timeout for the whole test instead of using dispatchTimeoutMs. " + + "Warning: the proposed replacement is not identical as it uses 'dispatchTimeoutMs' as the timeout for the whole test!", + ReplaceWith("this.runTest(timeout = dispatchTimeoutMs.milliseconds, testBody)", + "kotlin.time.Duration.Companion.milliseconds"), + DeprecationLevel.WARNING +) // Warning since 1.7.0, was experimental in 1.6.x public fun TestScope.runTest( - dispatchTimeoutMs: Long = DEFAULT_DISPATCH_TIMEOUT_MS, + dispatchTimeoutMs: Long, testBody: suspend TestScope.() -> Unit ): TestResult = asSpecificImplementation().let { it.enter() + @Suppress("DEPRECATION") createTestResult { - runTestCoroutine(it, dispatchTimeoutMs, TestScopeImpl::tryGetCompletionCause, testBody) { + runTestCoroutineLegacy(it, dispatchTimeoutMs.milliseconds, TestScopeImpl::tryGetCompletionCause, testBody) { backgroundScope.cancel() testScheduler.advanceUntilIdleOr { false } - it.leave() + it.legacyLeave() } } } @@ -191,17 +420,23 @@ internal object RunningInRunTest : CoroutineContext.Key<RunningInRunTest>, Corou internal const val DEFAULT_DISPATCH_TIMEOUT_MS = 60_000L /** + * The default timeout to use when running a test. + */ +internal val DEFAULT_TIMEOUT = 10.seconds + +/** * Run the [body][testBody] of the [test coroutine][coroutine], waiting for asynchronous completions for at most - * [dispatchTimeoutMs] milliseconds, and performing the [cleanup] procedure at the end. + * [dispatchTimeout] and performing the [cleanup] procedure at the end. * * [tryGetCompletionCause] is the [JobSupport.completionCause], which is passed explicitly because it is protected. * * The [cleanup] procedure may either throw [UncompletedCoroutinesError] to denote that child coroutines were leaked, or * return a list of uncaught exceptions that should be reported at the end of the test. */ -internal suspend fun <T: AbstractCoroutine<Unit>> CoroutineScope.runTestCoroutine( +@Deprecated("Used for support of legacy behavior") +internal suspend fun <T : AbstractCoroutine<Unit>> CoroutineScope.runTestCoroutineLegacy( coroutine: T, - dispatchTimeoutMs: Long, + dispatchTimeout: Duration, tryGetCompletionCause: T.() -> Throwable?, testBody: suspend T.() -> Unit, cleanup: () -> List<Throwable>, @@ -212,6 +447,8 @@ internal suspend fun <T: AbstractCoroutine<Unit>> CoroutineScope.runTestCoroutin testBody() } /** + * This is the legacy behavior, kept for now for compatibility only. + * * The general procedure here is as follows: * 1. Try running the work that the scheduler knows about, both background and foreground. * @@ -237,16 +474,22 @@ internal suspend fun <T: AbstractCoroutine<Unit>> CoroutineScope.runTestCoroutin scheduler.advanceUntilIdle() if (coroutine.isCompleted) { /* don't even enter `withTimeout`; this allows to use a timeout of zero to check that there are no - non-trivial dispatches. */ + non-trivial dispatches. */ completed = true continue } // in case progress depends on some background work, we need to keep spinning it. val backgroundWorkRunner = launch(CoroutineName("background work runner")) { while (true) { - scheduler.tryRunNextTaskUnless { !isActive } - // yield so that the `select` below has a chance to check if its conditions are fulfilled - yield() + val executedSomething = scheduler.tryRunNextTaskUnless { !isActive } + if (executedSomething) { + // yield so that the `select` below has a chance to finish successfully or time out + yield() + } else { + // no more tasks, we should suspend until there are some more. + // this doesn't interfere with the `select` below, because different channels are used. + scheduler.receiveDispatchEvent() + } } } try { @@ -255,11 +498,11 @@ internal suspend fun <T: AbstractCoroutine<Unit>> CoroutineScope.runTestCoroutin // observe that someone completed the test coroutine and leave without waiting for the timeout completed = true } - scheduler.onDispatchEvent { + scheduler.onDispatchEventForeground { // we received knowledge that `scheduler` observed a dispatch event, so we reset the timeout } - onTimeout(dispatchTimeoutMs) { - handleTimeout(coroutine, dispatchTimeoutMs, tryGetCompletionCause, cleanup) + onTimeout(dispatchTimeout) { + throw handleTimeout(coroutine, dispatchTimeout, tryGetCompletionCause, cleanup) } } } finally { @@ -273,21 +516,20 @@ internal suspend fun <T: AbstractCoroutine<Unit>> CoroutineScope.runTestCoroutin // it's normal that some jobs are not completed if the test body has failed, won't clutter the output emptyList() } - (listOf(exception) + exceptions).throwAll() + throwAll(exception, exceptions) } - cleanup().throwAll() + throwAll(null, cleanup()) } /** - * Invoked on timeout in [runTest]. Almost always just builds a nice [UncompletedCoroutinesError] and throws it. - * However, sometimes it detects that the coroutine completed, in which case it returns normally. + * Invoked on timeout in [runTest]. Just builds a nice [UncompletedCoroutinesError] and returns it. */ -private inline fun<T: AbstractCoroutine<Unit>> handleTimeout( +private inline fun <T : AbstractCoroutine<Unit>> handleTimeout( coroutine: T, - dispatchTimeoutMs: Long, + dispatchTimeout: Duration, tryGetCompletionCause: T.() -> Throwable?, cleanup: () -> List<Throwable>, -) { +): AssertionError { val uncaughtExceptions = try { cleanup() } catch (e: UncompletedCoroutinesError) { @@ -296,26 +538,48 @@ private inline fun<T: AbstractCoroutine<Unit>> handleTimeout( } val activeChildren = coroutine.children.filter { it.isActive }.toList() val completionCause = if (coroutine.isCancelled) coroutine.tryGetCompletionCause() else null - var message = "After waiting for $dispatchTimeoutMs ms" + var message = "After waiting for $dispatchTimeout" if (completionCause == null) message += ", the test coroutine is not completing" if (activeChildren.isNotEmpty()) message += ", there were active child jobs: $activeChildren" if (completionCause != null && activeChildren.isEmpty()) { - if (coroutine.isCompleted) - return - // TODO: can this really ever happen? - message += ", the test coroutine was not completed" + message += if (coroutine.isCompleted) + ", the test coroutine completed" + else + ", the test coroutine was not completed" } val error = UncompletedCoroutinesError(message) completionCause?.let { cause -> error.addSuppressed(cause) } uncaughtExceptions.forEach { error.addSuppressed(it) } - throw error + return error } -internal fun List<Throwable>.throwAll() { - firstOrNull()?.apply { - drop(1).forEach { addSuppressed(it) } - throw this +internal fun throwAll(head: Throwable?, other: List<Throwable>) { + if (head != null) { + other.forEach { head.addSuppressed(it) } + throw head + } else { + with(other) { + firstOrNull()?.apply { + drop(1).forEach { addSuppressed(it) } + throw this + } + } } } + +internal expect fun dumpCoroutines() + +@Deprecated( + "This is for binary compatibility with the `runTest` overload that existed at some point", + level = DeprecationLevel.HIDDEN +) +@JvmName("runTest\$default") +@Suppress("DEPRECATION", "UNUSED_PARAMETER") +public fun TestScope.runTestLegacy( + dispatchTimeoutMs: Long, + testBody: suspend TestScope.() -> Unit, + marker: Int, + unused2: Any?, +): TestResult = runTest(dispatchTimeoutMs = if (marker and 1 != 0) dispatchTimeoutMs else 60_000L, testBody) diff --git a/kotlinx-coroutines-test/common/src/TestCoroutineDispatchers.kt b/kotlinx-coroutines-test/common/src/TestCoroutineDispatchers.kt index e99fe8b1..3777cd26 100644 --- a/kotlinx-coroutines-test/common/src/TestCoroutineDispatchers.kt +++ b/kotlinx-coroutines-test/common/src/TestCoroutineDispatchers.kt @@ -137,7 +137,6 @@ private class UnconfinedTestDispatcherImpl( * * @see UnconfinedTestDispatcher for a dispatcher that is not confined to any particular thread. */ -@ExperimentalCoroutinesApi @Suppress("FunctionName") public fun StandardTestDispatcher( scheduler: TestCoroutineScheduler? = null, diff --git a/kotlinx-coroutines-test/common/src/TestCoroutineScheduler.kt b/kotlinx-coroutines-test/common/src/TestCoroutineScheduler.kt index e735c6d4..04e320d8 100644 --- a/kotlinx-coroutines-test/common/src/TestCoroutineScheduler.kt +++ b/kotlinx-coroutines-test/common/src/TestCoroutineScheduler.kt @@ -13,6 +13,7 @@ import kotlinx.coroutines.selects.* import kotlin.coroutines.* import kotlin.jvm.* import kotlin.time.* +import kotlin.time.Duration.Companion.milliseconds /** * This is a scheduler for coroutines used in tests, providing the delay-skipping behavior. @@ -26,7 +27,6 @@ import kotlin.time.* * virtual time as needed (via [advanceUntilIdle]), or run the tasks that are scheduled to run as soon as possible but * haven't yet been dispatched (via [runCurrent]). */ -@ExperimentalCoroutinesApi public class TestCoroutineScheduler : AbstractCoroutineContextElement(TestCoroutineScheduler), CoroutineContext.Element { @@ -49,6 +49,9 @@ public class TestCoroutineScheduler : AbstractCoroutineContextElement(TestCorout get() = synchronized(lock) { field } private set + /** A channel for notifying about the fact that a foreground work dispatch recently happened. */ + private val dispatchEventsForeground: Channel<Unit> = Channel(CONFLATED) + /** A channel for notifying about the fact that a dispatch recently happened. */ private val dispatchEvents: Channel<Unit> = Channel(CONFLATED) @@ -73,8 +76,8 @@ public class TestCoroutineScheduler : AbstractCoroutineContextElement(TestCorout val time = addClamping(currentTime, timeDeltaMillis) val event = TestDispatchEvent(dispatcher, count, time, marker as Any, isForeground) { isCancelled(marker) } events.addLast(event) - /** can't be moved above: otherwise, [onDispatchEvent] could consume the token sent here before there's - * actually anything in the event queue. */ + /** can't be moved above: otherwise, [onDispatchEventForeground] or [onDispatchEvent] could consume the + * token sent here before there's actually anything in the event queue. */ sendDispatchEvent(context) DisposableHandle { synchronized(lock) { @@ -97,7 +100,7 @@ public class TestCoroutineScheduler : AbstractCoroutineContextElement(TestCorout currentTime = event.time event } - event.dispatcher.processEvent(event.time, event.marker) + event.dispatcher.processEvent(event.marker) return true } @@ -105,11 +108,10 @@ public class TestCoroutineScheduler : AbstractCoroutineContextElement(TestCorout * Runs the enqueued tasks in the specified order, advancing the virtual time as needed until there are no more * tasks associated with the dispatchers linked to this scheduler. * - * A breaking change from [TestCoroutineDispatcher.advanceTimeBy] is that it no longer returns the total number of + * A breaking change from `TestCoroutineDispatcher.advanceTimeBy` is that it no longer returns the total number of * milliseconds by which the execution of this method has advanced the virtual time. If you want to recreate that * functionality, query [currentTime] before and after the execution to achieve the same result. */ - @ExperimentalCoroutinesApi public fun advanceUntilIdle(): Unit = advanceUntilIdleOr { events.none(TestDispatchEvent<*>::isForeground) } /** @@ -125,14 +127,13 @@ public class TestCoroutineScheduler : AbstractCoroutineContextElement(TestCorout /** * Runs the tasks that are scheduled to execute at this moment of virtual time. */ - @ExperimentalCoroutinesApi public fun runCurrent() { val timeMark = synchronized(lock) { currentTime } while (true) { val event = synchronized(lock) { events.removeFirstIf { it.time <= timeMark } ?: return } - event.dispatcher.processEvent(event.time, event.marker) + event.dispatcher.processEvent(event.marker) } } @@ -150,13 +151,21 @@ public class TestCoroutineScheduler : AbstractCoroutineContextElement(TestCorout * * Overflowing the target time used to lead to nothing being done, but will now run the tasks scheduled at up to * (but not including) [Long.MAX_VALUE]. * - * @throws IllegalStateException if passed a negative [delay][delayTimeMillis]. + * @throws IllegalArgumentException if passed a negative [delay][delayTimeMillis]. */ @ExperimentalCoroutinesApi - public fun advanceTimeBy(delayTimeMillis: Long) { - require(delayTimeMillis >= 0) { "Can not advance time by a negative delay: $delayTimeMillis" } + public fun advanceTimeBy(delayTimeMillis: Long): Unit = advanceTimeBy(delayTimeMillis.milliseconds) + + /** + * Moves the virtual clock of this dispatcher forward by [the specified amount][delayTime], running the + * scheduled tasks in the meantime. + * + * @throws IllegalArgumentException if passed a negative [delay][delayTime]. + */ + public fun advanceTimeBy(delayTime: Duration) { + require(!delayTime.isNegative()) { "Can not advance time by a negative delay: $delayTime" } val startingTime = currentTime - val targetTime = addClamping(startingTime, delayTimeMillis) + val targetTime = addClamping(startingTime, delayTime.inWholeMilliseconds) while (true) { val event = synchronized(lock) { val timeMark = currentTime @@ -173,7 +182,7 @@ public class TestCoroutineScheduler : AbstractCoroutineContextElement(TestCorout } } } - event.dispatcher.processEvent(event.time, event.marker) + event.dispatcher.processEvent(event.marker) } } @@ -191,21 +200,31 @@ public class TestCoroutineScheduler : AbstractCoroutineContextElement(TestCorout * [context] is the context in which the task will be dispatched. */ internal fun sendDispatchEvent(context: CoroutineContext) { + dispatchEvents.trySend(Unit) if (context[BackgroundWork] !== BackgroundWork) - dispatchEvents.trySend(Unit) + dispatchEventsForeground.trySend(Unit) } /** + * Waits for a notification about a dispatch event. + */ + internal suspend fun receiveDispatchEvent() = dispatchEvents.receive() + + /** * Consumes the knowledge that a dispatch event happened recently. */ internal val onDispatchEvent: SelectClause1<Unit> get() = dispatchEvents.onReceive /** + * Consumes the knowledge that a foreground work dispatch event happened recently. + */ + internal val onDispatchEventForeground: SelectClause1<Unit> get() = dispatchEventsForeground.onReceive + + /** * Returns the [TimeSource] representation of the virtual time of this scheduler. */ - @ExperimentalCoroutinesApi @ExperimentalTime - public val timeSource: TimeSource = object : AbstractLongTimeSource(DurationUnit.MILLISECONDS) { + public val timeSource: TimeSource.WithComparableMarks = object : AbstractLongTimeSource(DurationUnit.MILLISECONDS) { override fun read(): Long = currentTime } } diff --git a/kotlinx-coroutines-test/common/src/TestDispatcher.kt b/kotlinx-coroutines-test/common/src/TestDispatcher.kt index 348cc2f1..b027131c 100644 --- a/kotlinx-coroutines-test/common/src/TestDispatcher.kt +++ b/kotlinx-coroutines-test/common/src/TestDispatcher.kt @@ -7,6 +7,7 @@ package kotlinx.coroutines.test import kotlinx.coroutines.* import kotlin.coroutines.* import kotlin.jvm.* +import kotlin.time.* /** * A test dispatcher that can interface with a [TestCoroutineScheduler]. @@ -16,14 +17,13 @@ import kotlin.jvm.* * * [UnconfinedTestDispatcher] is a dispatcher that behaves like [Dispatchers.Unconfined] while allowing to control * the virtual time. */ -@ExperimentalCoroutinesApi -public abstract class TestDispatcher internal constructor() : CoroutineDispatcher(), Delay { +@Suppress("INVISIBLE_REFERENCE") +public abstract class TestDispatcher internal constructor() : CoroutineDispatcher(), Delay, DelayWithTimeoutDiagnostics { /** The scheduler that this dispatcher is linked to. */ - @ExperimentalCoroutinesApi public abstract val scheduler: TestCoroutineScheduler /** Notifies the dispatcher that it should process a single event marked with [marker] happening at time [time]. */ - internal fun processEvent(time: Long, marker: Any) { + internal fun processEvent(marker: Any) { check(marker is Runnable) marker.run() } @@ -31,12 +31,26 @@ public abstract class TestDispatcher internal constructor() : CoroutineDispatche /** @suppress */ override fun scheduleResumeAfterDelay(timeMillis: Long, continuation: CancellableContinuation<Unit>) { val timedRunnable = CancellableContinuationRunnable(continuation, this) - scheduler.registerEvent(this, timeMillis, timedRunnable, continuation.context, ::cancellableRunnableIsCancelled) + val handle = scheduler.registerEvent( + this, + timeMillis, + timedRunnable, + continuation.context, + ::cancellableRunnableIsCancelled + ) + continuation.disposeOnCancellation(handle) } /** @suppress */ override fun invokeOnTimeout(timeMillis: Long, block: Runnable, context: CoroutineContext): DisposableHandle = scheduler.registerEvent(this, timeMillis, block, context) { false } + + /** @suppress */ + @Suppress("CANNOT_OVERRIDE_INVISIBLE_MEMBER") + @Deprecated("Is only needed internally", level = DeprecationLevel.HIDDEN) + public override fun timeoutMessage(timeout: Duration): String = + "Timed out after $timeout of _virtual_ (kotlinx.coroutines.test) time. " + + "To use the real time, wrap 'withTimeout' in 'withContext(Dispatchers.Default.limitedParallelism(1))'" } /** diff --git a/kotlinx-coroutines-test/common/src/TestScope.kt b/kotlinx-coroutines-test/common/src/TestScope.kt index 15d48a2a..fa6e3930 100644 --- a/kotlinx-coroutines-test/common/src/TestScope.kt +++ b/kotlinx-coroutines-test/common/src/TestScope.kt @@ -40,21 +40,19 @@ import kotlin.time.* * paused by default, like [StandardTestDispatcher]. * * No access to the list of unhandled exceptions. */ -@ExperimentalCoroutinesApi public sealed interface TestScope : CoroutineScope { /** * The delay-skipping scheduler used by the test dispatchers running the code in this scope. */ - @ExperimentalCoroutinesApi public val testScheduler: TestCoroutineScheduler /** * A scope for background work. * * This scope is automatically cancelled when the test finishes. - * Additionally, while the coroutines in this scope are run as usual when - * using [advanceTimeBy] and [runCurrent], [advanceUntilIdle] will stop advancing the virtual time - * once only the coroutines in this scope are left unprocessed. + * The coroutines in this scope are run as usual when using [advanceTimeBy] and [runCurrent]. + * [advanceUntilIdle], on the other hand, will stop advancing the virtual time once only the coroutines in this + * scope are left unprocessed. * * Failures in coroutines in this scope do not terminate the test. * Instead, they are reported at the end of the test. @@ -82,7 +80,6 @@ public sealed interface TestScope : CoroutineScope { * } * ``` */ - @ExperimentalCoroutinesApi public val backgroundScope: CoroutineScope } @@ -124,12 +121,22 @@ public fun TestScope.runCurrent(): Unit = testScheduler.runCurrent() public fun TestScope.advanceTimeBy(delayTimeMillis: Long): Unit = testScheduler.advanceTimeBy(delayTimeMillis) /** + * Moves the virtual clock of this dispatcher forward by [the specified amount][delayTime], running the + * scheduled tasks in the meantime. + * + * @throws IllegalStateException if passed a negative [delay][delayTime]. + * @see TestCoroutineScheduler.advanceTimeBy + */ +@ExperimentalCoroutinesApi +public fun TestScope.advanceTimeBy(delayTime: Duration): Unit = testScheduler.advanceTimeBy(delayTime) + +/** * The [test scheduler][TestScope.testScheduler] as a [TimeSource]. * @see TestCoroutineScheduler.timeSource */ @ExperimentalCoroutinesApi @ExperimentalTime -public val TestScope.testTimeSource: TimeSource get() = testScheduler.timeSource +public val TestScope.testTimeSource: TimeSource.WithComparableMarks get() = testScheduler.timeSource /** * Creates a [TestScope]. @@ -156,7 +163,6 @@ public val TestScope.testTimeSource: TimeSource get() = testScheduler.timeSource * @throws IllegalArgumentException if [context] has an [CoroutineExceptionHandler] that is not an * [UncaughtExceptionCaptor]. */ -@ExperimentalCoroutinesApi @Suppress("FunctionName") public fun TestScope(context: CoroutineContext = EmptyCoroutineContext): TestScope { val ctxWithDispatcher = context.withDelaySkipping() @@ -220,6 +226,16 @@ internal class TestScopeImpl(context: CoroutineContext) : throw IllegalStateException("Only a single call to `runTest` can be performed during one test.") entered = true check(!finished) + /** the order is important: [reportException] is only guaranteed not to throw if [entered] is `true` but + * [finished] is `false`. + * However, we also want [uncaughtExceptions] to be queried after the callback is registered, + * because the exception collector will be able to report the exceptions that arrived before this test but + * after the previous one, and learning about such exceptions as soon is possible is nice. */ + @Suppress("INVISIBLE_REFERENCE", "INVISIBLE_MEMBER") + run { ensurePlatformExceptionHandlerLoaded(ExceptionCollector) } + if (catchNonTestRelatedExceptions) { + ExceptionCollector.addOnExceptionCallback(lock, this::reportException) + } uncaughtExceptions } if (exceptions.isNotEmpty()) { @@ -230,10 +246,21 @@ internal class TestScopeImpl(context: CoroutineContext) : } } + /** Called at the end of the test. May only be called once. Returns the list of caught unhandled exceptions. */ + fun leave(): List<Throwable> = synchronized(lock) { + check(entered && !finished) + /** After [finished] becomes `true`, it is no longer valid to have [reportException] as the callback. */ + ExceptionCollector.removeOnExceptionCallback(lock) + finished = true + uncaughtExceptions + } + /** Called at the end of the test. May only be called once. */ - fun leave(): List<Throwable> { + fun legacyLeave(): List<Throwable> { val exceptions = synchronized(lock) { check(entered && !finished) + /** After [finished] becomes `true`, it is no longer valid to have [reportException] as the callback. */ + ExceptionCollector.removeOnExceptionCallback(lock) finished = true uncaughtExceptions } @@ -287,7 +314,7 @@ internal fun TestScope.asSpecificImplementation(): TestScopeImpl = when (this) { } internal class UncaughtExceptionsBeforeTest : IllegalStateException( - "There were uncaught exceptions in coroutines launched from TestScope before the test started. Please avoid this," + + "There were uncaught exceptions before the test started. Please avoid this," + " as such exceptions are also reported in a platform-dependent manner so that they are not lost." ) @@ -296,3 +323,12 @@ internal class UncaughtExceptionsBeforeTest : IllegalStateException( */ @ExperimentalCoroutinesApi internal class UncompletedCoroutinesError(message: String) : AssertionError(message) + +/** + * A flag that controls whether [TestScope] should attempt to catch arbitrary exceptions flying through the system. + * If it is enabled, then any exception that is not caught by the user code will be reported as a test failure. + * By default, it is enabled, but some tests may want to disable it to test the behavior of the system when they have + * their own exception handling procedures. + */ +@PublishedApi +internal var catchNonTestRelatedExceptions: Boolean = true diff --git a/kotlinx-coroutines-test/common/src/internal/ExceptionCollector.kt b/kotlinx-coroutines-test/common/src/internal/ExceptionCollector.kt new file mode 100644 index 00000000..70fcb948 --- /dev/null +++ b/kotlinx-coroutines-test/common/src/internal/ExceptionCollector.kt @@ -0,0 +1,100 @@ +/* + * Copyright 2016-2022 JetBrains s.r.o. Use of this source code is governed by the Apache 2.0 license. + */ + +package kotlinx.coroutines.test.internal + +import kotlinx.coroutines.* +import kotlinx.coroutines.internal.* +import kotlin.coroutines.* + +/** + * If [addOnExceptionCallback] is called, the provided callback will be evaluated each time + * [handleCoroutineException] is executed and can't find a [CoroutineExceptionHandler] to + * process the exception. + * + * When a callback is registered once, even if it's later removed, the system starts to assume that + * other callbacks will eventually be registered, and so collects the exceptions. + * Once a new callback is registered, the collected exceptions are used with it. + * + * The callbacks in this object are the last resort before relying on platform-dependent + * ways to report uncaught exceptions from coroutines. + */ +internal object ExceptionCollector : AbstractCoroutineContextElement(CoroutineExceptionHandler), CoroutineExceptionHandler { + private val lock = SynchronizedObject() + private var enabled = false + private val unprocessedExceptions = mutableListOf<Throwable>() + private val callbacks = mutableMapOf<Any, (Throwable) -> Unit>() + + /** + * Registers [callback] to be executed when an uncaught exception happens. + * [owner] is a key by which to distinguish different callbacks. + */ + fun addOnExceptionCallback(owner: Any, callback: (Throwable) -> Unit) = synchronized(lock) { + enabled = true // never becomes `false` again + val previousValue = callbacks.put(owner, callback) + check(previousValue === null) + // try to process the exceptions using the newly-registered callback + unprocessedExceptions.forEach { reportException(it) } + unprocessedExceptions.clear() + } + + /** + * Unregisters the callback associated with [owner]. + */ + fun removeOnExceptionCallback(owner: Any) = synchronized(lock) { + if (enabled) { + val existingValue = callbacks.remove(owner) + check(existingValue !== null) + } + } + + /** + * Tries to handle the exception by propagating it to an interested consumer. + * Returns `true` if the exception does not need further processing. + * + * Doesn't throw. + */ + fun handleException(exception: Throwable): Boolean = synchronized(lock) { + if (!enabled) return false + if (reportException(exception)) return true + /** we don't return the result of the `add` function because we don't have a guarantee + * that a callback will eventually appear and collect the unprocessed exceptions, so + * we can't consider [exception] to be properly handled. */ + unprocessedExceptions.add(exception) + return false + } + + /** + * Try to report [exception] to the existing callbacks. + */ + private fun reportException(exception: Throwable): Boolean { + var executedACallback = false + for (callback in callbacks.values) { + callback(exception) + executedACallback = true + /** We don't leave the function here because we want to fan-out the exceptions to every interested consumer, + * it's not enough to have the exception processed by one of them. + * The reason is, it's less big of a deal to observe multiple concurrent reports of bad behavior than not + * to observe the report in the exact callback that is connected to that bad behavior. */ + } + return executedACallback + } + + @Suppress("INVISIBLE_MEMBER") + override fun handleException(context: CoroutineContext, exception: Throwable) { + if (handleException(exception)) { + throw ExceptionSuccessfullyProcessed + } + } + + override fun equals(other: Any?): Boolean = other is ExceptionCollector || other is ExceptionCollectorAsService +} + +/** + * A workaround for being unable to treat an object as a `ServiceLoader` service. + */ +internal class ExceptionCollectorAsService: CoroutineExceptionHandler by ExceptionCollector { + override fun equals(other: Any?): Boolean = other is ExceptionCollectorAsService || other is ExceptionCollector + override fun hashCode(): Int = ExceptionCollector.hashCode() +} diff --git a/kotlinx-coroutines-test/common/src/internal/TestMainDispatcher.kt b/kotlinx-coroutines-test/common/src/internal/TestMainDispatcher.kt index 24e093be..411699b9 100644 --- a/kotlinx-coroutines-test/common/src/internal/TestMainDispatcher.kt +++ b/kotlinx-coroutines-test/common/src/internal/TestMainDispatcher.kt @@ -61,29 +61,32 @@ internal class TestMainDispatcher(delegate: CoroutineDispatcher): * next modification. */ private class NonConcurrentlyModifiable<T>(initialValue: T, private val name: String) { + private val reader: AtomicRef<Throwable?> = atomic(null) // last reader to attempt access private val readers = atomic(0) // number of concurrent readers - private val isWriting = atomic(false) // a modification is happening currently + private val writer: AtomicRef<Throwable?> = atomic(null) // writer currently performing value modification private val exceptionWhenReading: AtomicRef<Throwable?> = atomic(null) // exception from reading private val _value = atomic(initialValue) // the backing field for the value - private fun concurrentWW() = IllegalStateException("$name is modified concurrently") - private fun concurrentRW() = IllegalStateException("$name is used concurrently with setting it") + private fun concurrentWW(location: Throwable) = IllegalStateException("$name is modified concurrently", location) + private fun concurrentRW(location: Throwable) = IllegalStateException("$name is used concurrently with setting it", location) var value: T get() { + reader.value = Throwable("reader location") readers.incrementAndGet() - if (isWriting.value) exceptionWhenReading.value = concurrentRW() + writer.value?.let { exceptionWhenReading.value = concurrentRW(it) } val result = _value.value readers.decrementAndGet() return result } set(value) { exceptionWhenReading.getAndSet(null)?.let { throw it } - if (readers.value != 0) throw concurrentRW() - if (!isWriting.compareAndSet(expect = false, update = true)) throw concurrentWW() + if (readers.value != 0) reader.value?.let { throw concurrentRW(it) } + val writerLocation = Throwable("other writer location") + writer.getAndSet(writerLocation)?.let { throw concurrentWW(it) } _value.value = value - isWriting.value = false - if (readers.value != 0) throw concurrentRW() + writer.compareAndSet(writerLocation, null) + if (readers.value != 0) reader.value?.let { throw concurrentRW(it) } } } } diff --git a/kotlinx-coroutines-test/common/test/Helpers.kt b/kotlinx-coroutines-test/common/test/Helpers.kt index 98375b09..345c66f9 100644 --- a/kotlinx-coroutines-test/common/test/Helpers.kt +++ b/kotlinx-coroutines-test/common/test/Helpers.kt @@ -31,9 +31,32 @@ inline fun <T> assertRunsFast(timeout: Duration, block: () -> T): T { inline fun <T> assertRunsFast(block: () -> T): T = assertRunsFast(2.seconds, block) /** - * Passes [test] as an argument to [block], but as a function returning not a [TestResult] but [Unit]. + * Runs [test], and then invokes [block], passing to it the lambda that functionally behaves + * the same way [test] does. */ -expect fun testResultMap(block: (() -> Unit) -> Unit, test: () -> TestResult): TestResult +fun testResultMap(block: (() -> Unit) -> Unit, test: () -> TestResult): TestResult = testResultChain( + block = test, + after = { + block { it.getOrThrow() } + createTestResult { } + } +) + +/** + * Chains together [block] and [after], passing the result of [block] to [after]. + */ +expect fun testResultChain(block: () -> TestResult, after: (Result<Unit>) -> TestResult): TestResult + +fun testResultChain(vararg chained: (Result<Unit>) -> TestResult): TestResult = + if (chained.isEmpty()) { + createTestResult { } + } else { + testResultChain(block = { + chained[0](Result.success(Unit)) + }) { + testResultChain(*chained.drop(1).toTypedArray()) + } + } class TestException(message: String? = null): Exception(message) diff --git a/kotlinx-coroutines-test/common/test/RunTestTest.kt b/kotlinx-coroutines-test/common/test/RunTestTest.kt index 1430d830..da2bdcfc 100644 --- a/kotlinx-coroutines-test/common/test/RunTestTest.kt +++ b/kotlinx-coroutines-test/common/test/RunTestTest.kt @@ -9,6 +9,8 @@ import kotlinx.coroutines.internal.* import kotlinx.coroutines.flow.* import kotlin.coroutines.* import kotlin.test.* +import kotlin.time.* +import kotlin.time.Duration.Companion.milliseconds class RunTestTest { @@ -52,7 +54,7 @@ class RunTestTest { /** Tests that even the dispatch timeout of `0` is fine if all the dispatches go through the same scheduler. */ @Test - fun testRunTestWithZeroTimeoutWithControlledDispatches() = runTest(dispatchTimeoutMs = 0) { + fun testRunTestWithZeroDispatchTimeoutWithControlledDispatches() = runTest(dispatchTimeoutMs = 0) { // below is some arbitrary concurrent code where all dispatches go through the same scheduler. launch { delay(2000) @@ -69,28 +71,38 @@ class RunTestTest { deferred.await() } - /** Tests that a dispatch timeout of `0` will fail the test if there are some dispatches outside the scheduler. */ + /** Tests that too low of a dispatch timeout causes crashes. */ @Test - @NoNative // TODO: timeout leads to `Cannot execute task because event loop was shut down` on Native - fun testRunTestWithZeroTimeoutWithUncontrolledDispatches() = testResultMap({ fn -> - assertFailsWith<UncompletedCoroutinesError> { fn() } + fun testRunTestWithSmallDispatchTimeout() = testResultMap({ fn -> + try { + fn() + fail("shouldn't be reached") + } catch (e: Throwable) { + assertIs<UncompletedCoroutinesError>(e) + } }) { - runTest(dispatchTimeoutMs = 0) { + runTest(dispatchTimeoutMs = 100) { withContext(Dispatchers.Default) { - delay(10) + delay(10000) 3 } fail("shouldn't be reached") } } - /** Tests that too low of a dispatch timeout causes crashes. */ + /** + * Tests that [runTest] times out after the specified time. + */ @Test - @NoNative // TODO: timeout leads to `Cannot execute task because event loop was shut down` on Native fun testRunTestWithSmallTimeout() = testResultMap({ fn -> - assertFailsWith<UncompletedCoroutinesError> { fn() } + try { + fn() + fail("shouldn't be reached") + } catch (e: Throwable) { + assertIs<UncompletedCoroutinesError>(e) + } }) { - runTest(dispatchTimeoutMs = 100) { + runTest(timeout = 100.milliseconds) { withContext(Dispatchers.Default) { delay(10000) 3 @@ -99,6 +111,27 @@ class RunTestTest { } } + /** Tests that [runTest] times out after the specified time, even if the test framework always knows the test is + * still doing something. */ + @Test + fun testRunTestWithSmallTimeoutAndManyDispatches() = testResultMap({ fn -> + try { + fn() + fail("shouldn't be reached") + } catch (e: Throwable) { + assertIs<UncompletedCoroutinesError>(e) + } + }) { + runTest(timeout = 100.milliseconds) { + while (true) { + withContext(Dispatchers.Default) { + delay(10) + 3 + } + } + } + } + /** Tests that, on timeout, the names of the active coroutines are listed, * whereas the names of the completed ones are not. */ @Test @@ -112,7 +145,7 @@ class RunTestTest { it() fail("unreached") } catch (e: UncompletedCoroutinesError) { - assertTrue((e.message ?: "").contains(name1)) + assertContains(e.message ?: "", name1) assertFalse((e.message ?: "").contains(name2)) } }) { @@ -135,26 +168,33 @@ class RunTestTest { } catch (e: UncompletedCoroutinesError) { @Suppress("INVISIBLE_MEMBER") val suppressed = unwrap(e).suppressedExceptions - assertEquals(1, suppressed.size) + assertEquals(1, suppressed.size, "$suppressed") assertIs<TestException>(suppressed[0]).also { assertEquals("A", it.message) } } }) { - runTest(dispatchTimeoutMs = 10) { - launch { - withContext(NonCancellable) { - awaitCancellation() + runTest(timeout = 10.milliseconds) { + launch(start = CoroutineStart.UNDISPATCHED) { + withContext(NonCancellable + Dispatchers.Default) { + delay(100.milliseconds) } } - yield() throw TestException("A") } } /** Tests that real delays can be accounted for with a large enough dispatch timeout. */ @Test - fun testRunTestWithLargeTimeout() = runTest(dispatchTimeoutMs = 5000) { + fun testRunTestWithLargeDispatchTimeout() = runTest(dispatchTimeoutMs = 5000) { + withContext(Dispatchers.Default) { + delay(50) + } + } + + /** Tests that delays can be accounted for with a large enough timeout. */ + @Test + fun testRunTestWithLargeTimeout() = runTest(timeout = 5000.milliseconds) { withContext(Dispatchers.Default) { delay(50) } @@ -162,7 +202,6 @@ class RunTestTest { /** Tests uncaught exceptions being suppressed by the dispatch timeout error. */ @Test - @NoNative // TODO: timeout leads to `Cannot execute task because event loop was shut down` on Native fun testRunTestTimingOutAndThrowing() = testResultMap({ fn -> try { fn() @@ -170,13 +209,13 @@ class RunTestTest { } catch (e: UncompletedCoroutinesError) { @Suppress("INVISIBLE_MEMBER") val suppressed = unwrap(e).suppressedExceptions - assertEquals(1, suppressed.size) + assertEquals(1, suppressed.size, "$suppressed") assertIs<TestException>(suppressed[0]).also { assertEquals("A", it.message) } } }) { - runTest(dispatchTimeoutMs = 1) { + runTest(timeout = 100.milliseconds) { coroutineContext[CoroutineExceptionHandler]!!.handleException(coroutineContext, TestException("A")) withContext(Dispatchers.Default) { delay(10000) @@ -341,7 +380,7 @@ class RunTestTest { } } - /** Tests that [TestCoroutineScope.runTest] does not inherit the exception handler and works. */ + /** Tests that [TestScope.runTest] does not inherit the exception handler and works. */ @Test fun testScopeRunTestExceptionHandler(): TestResult { val scope = TestScope() @@ -366,7 +405,7 @@ class RunTestTest { * The test will hang if this is not the case. */ @Test - fun testCoroutineCompletingWithoutDispatch() = runTest(dispatchTimeoutMs = Long.MAX_VALUE) { + fun testCoroutineCompletingWithoutDispatch() = runTest(timeout = Duration.INFINITE) { launch(Dispatchers.Default) { delay(100) } } } diff --git a/kotlinx-coroutines-test/common/test/StandardTestDispatcherTest.kt b/kotlinx-coroutines-test/common/test/StandardTestDispatcherTest.kt index d66be9bd..280d6685 100644 --- a/kotlinx-coroutines-test/common/test/StandardTestDispatcherTest.kt +++ b/kotlinx-coroutines-test/common/test/StandardTestDispatcherTest.kt @@ -20,7 +20,7 @@ class StandardTestDispatcherTest: OrderedExecutionTestBase() { @AfterTest fun cleanup() { scope.runCurrent() - assertEquals(listOf(), scope.asSpecificImplementation().leave()) + assertEquals(listOf(), scope.asSpecificImplementation().legacyLeave()) } /** Tests that the [StandardTestDispatcher] follows an execution order similar to `runBlocking`. */ @@ -64,7 +64,6 @@ class StandardTestDispatcherTest: OrderedExecutionTestBase() { /** Tests that the [TestCoroutineScheduler] used for [Dispatchers.Main] gets used by default. */ @Test - @NoNative fun testSchedulerReuse() { val dispatcher1 = StandardTestDispatcher() Dispatchers.setMain(dispatcher1) @@ -76,4 +75,4 @@ class StandardTestDispatcherTest: OrderedExecutionTestBase() { } } -}
\ No newline at end of file +} diff --git a/kotlinx-coroutines-test/common/test/TestCoroutineSchedulerTest.kt b/kotlinx-coroutines-test/common/test/TestCoroutineSchedulerTest.kt index d050e9c8..4aec7731 100644 --- a/kotlinx-coroutines-test/common/test/TestCoroutineSchedulerTest.kt +++ b/kotlinx-coroutines-test/common/test/TestCoroutineSchedulerTest.kt @@ -8,6 +8,7 @@ import kotlinx.coroutines.* import kotlin.test.* import kotlin.time.* import kotlin.time.Duration.Companion.seconds +import kotlin.time.Duration.Companion.milliseconds class TestCoroutineSchedulerTest { /** Tests that `TestCoroutineScheduler` attempts to detect if there are several instances of it. */ @@ -28,7 +29,7 @@ class TestCoroutineSchedulerTest { delay(15) entered = true } - testScheduler.advanceTimeBy(15) + testScheduler.advanceTimeBy(15.milliseconds) assertFalse(entered) testScheduler.runCurrent() assertTrue(entered) @@ -39,7 +40,7 @@ class TestCoroutineSchedulerTest { fun testAdvanceTimeByWithNegativeDelay() { val scheduler = TestCoroutineScheduler() assertFailsWith<IllegalArgumentException> { - scheduler.advanceTimeBy(-1) + scheduler.advanceTimeBy((-1).milliseconds) } } @@ -65,7 +66,7 @@ class TestCoroutineSchedulerTest { assertEquals(Long.MAX_VALUE - 1, currentTime) enteredNearInfinity = true } - testScheduler.advanceTimeBy(Long.MAX_VALUE) + testScheduler.advanceTimeBy(Duration.INFINITE) assertFalse(enteredInfinity) assertTrue(enteredNearInfinity) assertEquals(Long.MAX_VALUE, currentTime) @@ -95,10 +96,10 @@ class TestCoroutineSchedulerTest { } assertEquals(1, stage) assertEquals(0, currentTime) - advanceTimeBy(2_000) + advanceTimeBy(2.seconds) assertEquals(3, stage) assertEquals(2_000, currentTime) - advanceTimeBy(2) + advanceTimeBy(2.milliseconds) assertEquals(4, stage) assertEquals(2_002, currentTime) } @@ -120,11 +121,11 @@ class TestCoroutineSchedulerTest { delay(1) stage += 10 } - testScheduler.advanceTimeBy(1) + testScheduler.advanceTimeBy(1.milliseconds) assertEquals(0, stage) runCurrent() assertEquals(2, stage) - testScheduler.advanceTimeBy(1) + testScheduler.advanceTimeBy(1.milliseconds) assertEquals(2, stage) runCurrent() assertEquals(22, stage) @@ -143,10 +144,10 @@ class TestCoroutineSchedulerTest { delay(SLOW) stage = 3 } - scheduler.advanceTimeBy(SLOW) + scheduler.advanceTimeBy(SLOW.milliseconds) stage = 2 } - scheduler.advanceTimeBy(SLOW) + scheduler.advanceTimeBy(SLOW.milliseconds) assertEquals(1, stage) scheduler.runCurrent() assertEquals(2, stage) @@ -249,7 +250,7 @@ class TestCoroutineSchedulerTest { } } advanceUntilIdle() - asSpecificImplementation().leave().throwAll() + throwAll(null, asSpecificImplementation().legacyLeave()) if (timesOut) assertTrue(caughtException) else @@ -314,10 +315,14 @@ class TestCoroutineSchedulerTest { @ExperimentalTime fun testAdvanceTimeSource() = runTest { val expected = 1.seconds + val before = testTimeSource.markNow() val actual = testTimeSource.measureTime { delay(expected) } assertEquals(expected, actual) + val after = testTimeSource.markNow() + assertTrue(before < after) + assertEquals(expected, after - before) } private fun forTestDispatchers(block: (TestDispatcher) -> Unit): Unit = diff --git a/kotlinx-coroutines-test/common/test/TestDispatchersTest.kt b/kotlinx-coroutines-test/common/test/TestDispatchersTest.kt index bcf016b3..da48e7f4 100644 --- a/kotlinx-coroutines-test/common/test/TestDispatchersTest.kt +++ b/kotlinx-coroutines-test/common/test/TestDispatchersTest.kt @@ -8,7 +8,6 @@ import kotlinx.coroutines.test.internal.* import kotlin.coroutines.* import kotlin.test.* -@NoNative class TestDispatchersTest: OrderedExecutionTestBase() { @BeforeTest diff --git a/kotlinx-coroutines-test/common/test/TestScopeTest.kt b/kotlinx-coroutines-test/common/test/TestScopeTest.kt index 4138ca05..433faef7 100644 --- a/kotlinx-coroutines-test/common/test/TestScopeTest.kt +++ b/kotlinx-coroutines-test/common/test/TestScopeTest.kt @@ -9,6 +9,7 @@ import kotlinx.coroutines.channels.* import kotlinx.coroutines.flow.* import kotlin.coroutines.* import kotlin.test.* +import kotlin.time.Duration.Companion.milliseconds class TestScopeTest { /** Tests failing to create a [TestScope] with incorrect contexts. */ @@ -54,7 +55,6 @@ class TestScopeTest { /** Part of [testCreateProvidesScheduler], disabled for Native */ @Test - @NoNative fun testCreateReusesScheduler() { // Reuses the scheduler of `Dispatchers.Main` run { @@ -96,7 +96,7 @@ class TestScopeTest { } assertFalse(result) scope.asSpecificImplementation().enter() - assertFailsWith<UncompletedCoroutinesError> { scope.asSpecificImplementation().leave() } + assertFailsWith<UncompletedCoroutinesError> { scope.asSpecificImplementation().legacyLeave() } assertFalse(result) } @@ -112,7 +112,7 @@ class TestScopeTest { } assertFalse(result) scope.asSpecificImplementation().enter() - assertFailsWith<UncompletedCoroutinesError> { scope.asSpecificImplementation().leave() } + assertFailsWith<UncompletedCoroutinesError> { scope.asSpecificImplementation().legacyLeave() } assertFalse(result) } @@ -129,7 +129,7 @@ class TestScopeTest { job.cancel() assertFalse(result) scope.asSpecificImplementation().enter() - assertFailsWith<UncompletedCoroutinesError> { scope.asSpecificImplementation().leave() } + assertFailsWith<UncompletedCoroutinesError> { scope.asSpecificImplementation().legacyLeave() } assertFalse(result) } @@ -163,7 +163,7 @@ class TestScopeTest { launch(SupervisorJob()) { throw TestException("y") } launch(SupervisorJob()) { throw TestException("z") } runCurrent() - val e = asSpecificImplementation().leave() + val e = asSpecificImplementation().legacyLeave() assertEquals(3, e.size) assertEquals("x", e[0].message) assertEquals("y", e[1].message) @@ -250,7 +250,7 @@ class TestScopeTest { assertEquals(1, j) } job.join() - advanceTimeBy(199) // should work the same for the background tasks + advanceTimeBy(199.milliseconds) // should work the same for the background tasks assertEquals(2, i) assertEquals(4, j) advanceUntilIdle() // once again, should do nothing @@ -378,7 +378,7 @@ class TestScopeTest { } }) { - runTest(dispatchTimeoutMs = 100) { + runTest(timeout = 100.milliseconds) { backgroundScope.launch { while (true) { yield() @@ -408,7 +408,7 @@ class TestScopeTest { } }) { - runTest(UnconfinedTestDispatcher(), dispatchTimeoutMs = 100) { + runTest(UnconfinedTestDispatcher(), timeout = 100.milliseconds) { /** * Having a coroutine like this will still cause the test to hang: backgroundScope.launch { @@ -477,6 +477,76 @@ class TestScopeTest { } } + /** + * Tests that [TestScope.withTimeout] notifies the programmer about using the virtual time. + */ + @Test + fun testTimingOutWithVirtualTimeMessage() = runTest { + try { + withTimeout(1_000_000) { + Channel<Unit>().receive() + } + } catch (e: TimeoutCancellationException) { + assertContains(e.message!!, "virtual") + } + } + + /* + * Tests that the [TestScope] exception reporting mechanism will report the exceptions that happen between + * different tests. + * + * This test must be ran manually, because such exceptions still go through the global exception handler + * (as there's no guarantee that another test will happen), and the global exception handler will + * log the exceptions or, on Native, crash the test suite. + */ + @Test + @Ignore + fun testReportingStrayUncaughtExceptionsBetweenTests() { + val thrown = TestException("x") + testResultChain({ + // register a handler for uncaught exceptions + runTest { } + }, { + GlobalScope.launch(start = CoroutineStart.UNDISPATCHED) { + throw thrown + } + runTest { + fail("unreached") + } + }, { + // this `runTest` will not report the exception + runTest { + when (val exception = it.exceptionOrNull()) { + is UncaughtExceptionsBeforeTest -> { + assertEquals(1, exception.suppressedExceptions.size) + assertSame(exception.suppressedExceptions[0], thrown) + } + else -> fail("unexpected exception: $exception") + } + } + }) + } + + /** + * Tests that the uncaught exceptions that happen during the test are reported. + */ + @Test + fun testReportingStrayUncaughtExceptionsDuringTest(): TestResult { + val thrown = TestException("x") + return testResultChain({ _ -> + runTest { + val job = launch(Dispatchers.Default + NonCancellable) { + throw thrown + } + job.join() + } + }, { + runTest { + assertEquals(thrown, it.exceptionOrNull()) + } + }) + } + companion object { internal val invalidContexts = listOf( Dispatchers.Default, // not a [TestDispatcher] diff --git a/kotlinx-coroutines-test/common/test/UnconfinedTestDispatcherTest.kt b/kotlinx-coroutines-test/common/test/UnconfinedTestDispatcherTest.kt index ee63e6d1..23ff0ac1 100644 --- a/kotlinx-coroutines-test/common/test/UnconfinedTestDispatcherTest.kt +++ b/kotlinx-coroutines-test/common/test/UnconfinedTestDispatcherTest.kt @@ -153,7 +153,6 @@ class UnconfinedTestDispatcherTest { /** Tests that the [TestCoroutineScheduler] used for [Dispatchers.Main] gets used by default. */ @Test - @NoNative fun testSchedulerReuse() { val dispatcher1 = StandardTestDispatcher() Dispatchers.setMain(dispatcher1) @@ -165,4 +164,4 @@ class UnconfinedTestDispatcherTest { } } -}
\ No newline at end of file +} diff --git a/kotlinx-coroutines-test/js/src/TestBuilders.kt b/kotlinx-coroutines-test/js/src/TestBuilders.kt index 9da91ffc..97c9da0e 100644 --- a/kotlinx-coroutines-test/js/src/TestBuilders.kt +++ b/kotlinx-coroutines-test/js/src/TestBuilders.kt @@ -13,3 +13,5 @@ internal actual fun createTestResult(testProcedure: suspend CoroutineScope.() -> GlobalScope.promise { testProcedure() } + +internal actual fun dumpCoroutines() { } diff --git a/kotlinx-coroutines-test/js/test/Helpers.kt b/kotlinx-coroutines-test/js/test/Helpers.kt index 5f19d1ac..5fd0291c 100644 --- a/kotlinx-coroutines-test/js/test/Helpers.kt +++ b/kotlinx-coroutines-test/js/test/Helpers.kt @@ -1,20 +1,17 @@ /* - * Copyright 2016-2021 JetBrains s.r.o. Use of this source code is governed by the Apache 2.0 license. + * Copyright 2016-2022 JetBrains s.r.o. Use of this source code is governed by the Apache 2.0 license. */ package kotlinx.coroutines.test import kotlin.test.* -actual fun testResultMap(block: (() -> Unit) -> Unit, test: () -> TestResult): TestResult = - test().then( +actual fun testResultChain(block: () -> TestResult, after: (Result<Unit>) -> TestResult): TestResult = + block().then( { - block { - } + after(Result.success(Unit)) }, { - block { - throw it - } + after(Result.failure(it)) }) actual typealias NoJs = Ignore diff --git a/kotlinx-coroutines-test/jvm/resources/META-INF/services/kotlinx.coroutines.CoroutineExceptionHandler b/kotlinx-coroutines-test/jvm/resources/META-INF/services/kotlinx.coroutines.CoroutineExceptionHandler new file mode 100644 index 00000000..c9aaec2e --- /dev/null +++ b/kotlinx-coroutines-test/jvm/resources/META-INF/services/kotlinx.coroutines.CoroutineExceptionHandler @@ -0,0 +1 @@ +kotlinx.coroutines.test.internal.ExceptionCollectorAsService diff --git a/kotlinx-coroutines-test/jvm/src/TestBuildersJvm.kt b/kotlinx-coroutines-test/jvm/src/TestBuildersJvm.kt index 06fbe810..0521fd22 100644 --- a/kotlinx-coroutines-test/jvm/src/TestBuildersJvm.kt +++ b/kotlinx-coroutines-test/jvm/src/TestBuildersJvm.kt @@ -4,6 +4,7 @@ package kotlinx.coroutines.test import kotlinx.coroutines.* +import kotlinx.coroutines.debug.internal.* @Suppress("ACTUAL_WITHOUT_EXPECT") public actual typealias TestResult = Unit @@ -13,3 +14,16 @@ internal actual fun createTestResult(testProcedure: suspend CoroutineScope.() -> testProcedure() } } + +internal actual fun dumpCoroutines() { + @Suppress("INVISIBLE_REFERENCE", "INVISIBLE_MEMBER") + if (DebugProbesImpl.isInstalled) { + DebugProbesImpl.install() + try { + DebugProbesImpl.dumpCoroutines(System.err) + System.err.flush() + } finally { + DebugProbesImpl.uninstall() + } + } +} diff --git a/kotlinx-coroutines-test/jvm/src/migration/DelayController.kt b/kotlinx-coroutines-test/jvm/src/migration/DelayController.kt index 3ccf2cad..ab84da1c 100644 --- a/kotlinx-coroutines-test/jvm/src/migration/DelayController.kt +++ b/kotlinx-coroutines-test/jvm/src/migration/DelayController.kt @@ -1,7 +1,7 @@ /* * Copyright 2016-2021 JetBrains s.r.o. Use of this source code is governed by the Apache 2.0 license. */ -@file:Suppress("DEPRECATION") +@file:Suppress("DEPRECATION_ERROR") package kotlinx.coroutines.test @@ -21,7 +21,7 @@ import kotlinx.coroutines.* @ExperimentalCoroutinesApi @Deprecated( "Use `TestCoroutineScheduler` to control virtual time.", - level = DeprecationLevel.WARNING + level = DeprecationLevel.ERROR ) // Since 1.6.0, ERROR in 1.7.0 and removed as experimental in 1.8.0 public interface DelayController { @@ -111,7 +111,7 @@ public interface DelayController { */ @Deprecated( "Please use a dispatcher that is paused by default, like `StandardTestDispatcher`.", - level = DeprecationLevel.WARNING + level = DeprecationLevel.ERROR ) // Since 1.6.0, ERROR in 1.7.0 and removed as experimental in 1.8.0 public suspend fun pauseDispatcher(block: suspend () -> Unit) @@ -124,7 +124,7 @@ public interface DelayController { */ @Deprecated( "Please use a dispatcher that is paused by default, like `StandardTestDispatcher`.", - level = DeprecationLevel.WARNING + level = DeprecationLevel.ERROR ) // Since 1.6.0, ERROR in 1.7.0 and removed as experimental in 1.8.0 public fun pauseDispatcher() @@ -138,7 +138,7 @@ public interface DelayController { */ @Deprecated( "Please use a dispatcher that is paused by default, like `StandardTestDispatcher`.", - level = DeprecationLevel.WARNING + level = DeprecationLevel.ERROR ) // Since 1.6.0, ERROR in 1.7.0 and removed as experimental in 1.8.0 public fun resumeDispatcher() @@ -151,7 +151,7 @@ internal interface SchedulerAsDelayController : DelayController { @Deprecated( "This property delegates to the test scheduler, which may cause confusing behavior unless made explicit.", ReplaceWith("this.scheduler.currentTime"), - level = DeprecationLevel.WARNING + level = DeprecationLevel.ERROR ) // Since 1.6.0, ERROR in 1.7.0 and removed as experimental in 1.8.0 override val currentTime: Long @@ -162,7 +162,7 @@ internal interface SchedulerAsDelayController : DelayController { @Deprecated( "This function delegates to the test scheduler, which may cause confusing behavior unless made explicit.", ReplaceWith("this.scheduler.apply { advanceTimeBy(delayTimeMillis); runCurrent() }"), - level = DeprecationLevel.WARNING + level = DeprecationLevel.ERROR ) // Since 1.6.0, ERROR in 1.7.0 and removed as experimental in 1.8.0 override fun advanceTimeBy(delayTimeMillis: Long): Long { @@ -176,7 +176,7 @@ internal interface SchedulerAsDelayController : DelayController { @Deprecated( "This function delegates to the test scheduler, which may cause confusing behavior unless made explicit.", ReplaceWith("this.scheduler.advanceUntilIdle()"), - level = DeprecationLevel.WARNING + level = DeprecationLevel.ERROR ) // Since 1.6.0, ERROR in 1.7.0 and removed as experimental in 1.8.0 override fun advanceUntilIdle(): Long { @@ -189,7 +189,7 @@ internal interface SchedulerAsDelayController : DelayController { @Deprecated( "This function delegates to the test scheduler, which may cause confusing behavior unless made explicit.", ReplaceWith("this.scheduler.runCurrent()"), - level = DeprecationLevel.WARNING + level = DeprecationLevel.ERROR ) // Since 1.6.0, ERROR in 1.7.0 and removed as experimental in 1.8.0 override fun runCurrent(): Unit = scheduler.runCurrent() diff --git a/kotlinx-coroutines-test/jvm/src/migration/TestBuildersDeprecated.kt b/kotlinx-coroutines-test/jvm/src/migration/TestBuildersDeprecated.kt index eabdffb2..7a98fd1d 100644 --- a/kotlinx-coroutines-test/jvm/src/migration/TestBuildersDeprecated.kt +++ b/kotlinx-coroutines-test/jvm/src/migration/TestBuildersDeprecated.kt @@ -10,6 +10,7 @@ package kotlinx.coroutines.test import kotlinx.coroutines.* import kotlin.coroutines.* import kotlin.jvm.* +import kotlin.time.Duration.Companion.milliseconds /** * Executes a [testBody] inside an immediate execution dispatcher. @@ -49,11 +50,13 @@ import kotlin.jvm.* * then they must implement [DelayController] and [TestCoroutineExceptionHandler] respectively. * @param testBody The code of the unit-test. */ -@Deprecated("Use `runTest` instead to support completing from other dispatchers. " + - "Please see the migration guide for details: " + - "https://github.com/Kotlin/kotlinx.coroutines/blob/master/kotlinx-coroutines-test/MIGRATION.md", - level = DeprecationLevel.WARNING) -// Since 1.6.0, ERROR in 1.7.0 and removed as experimental in 1.8.0 +@Deprecated( + "Use `runTest` instead to support completing from other dispatchers. " + + "Please see the migration guide for details: " + + "https://github.com/Kotlin/kotlinx.coroutines/blob/master/kotlinx-coroutines-test/MIGRATION.md", + level = DeprecationLevel.WARNING +) +// Since 1.6.0, kept as warning in 1.7.0, ERROR in 1.8.0 and removed as experimental in 1.9.0 public fun runBlockingTest( context: CoroutineContext = EmptyCoroutineContext, testBody: suspend TestCoroutineScope.() -> Unit @@ -74,7 +77,7 @@ public fun runBlockingTest( * A version of [runBlockingTest] that works with [TestScope]. */ @Deprecated("Use `runTest` instead to support completing from other dispatchers.", level = DeprecationLevel.WARNING) -// Since 1.6.0, ERROR in 1.7.0 and removed as experimental in 1.8.0 +// Since 1.6.0, kept as warning in 1.7.0, ERROR in 1.8.0 and removed as experimental in 1.9.0 public fun runBlockingTestOnTestScope( context: CoroutineContext = EmptyCoroutineContext, testBody: suspend TestScope.() -> Unit @@ -90,20 +93,20 @@ public fun runBlockingTestOnTestScope( val throwable = try { scope.getCompletionExceptionOrNull() } catch (e: IllegalStateException) { - null // the deferred was not completed yet; `scope.leave()` should complain then about unfinished jobs + null // the deferred was not completed yet; `scope.legacyLeave()` should complain then about unfinished jobs } scope.backgroundScope.cancel() scope.testScheduler.advanceUntilIdleOr { false } throwable?.let { val exceptions = try { - scope.leave() + scope.legacyLeave() } catch (e: UncompletedCoroutinesError) { listOf() } - (listOf(it) + exceptions).throwAll() + throwAll(it, exceptions) return } - scope.leave().throwAll() + throwAll(null, scope.legacyLeave()) val jobs = completeContext.activeJobs() - startJobs if (jobs.isNotEmpty()) throw UncompletedCoroutinesError("Some jobs were not completed at the end of the test: $jobs") @@ -117,11 +120,13 @@ public fun runBlockingTestOnTestScope( * [migration guide](https://github.com/Kotlin/kotlinx.coroutines/blob/master/kotlinx-coroutines-test/MIGRATION.md) * for an instruction on how to update the code for the new API. */ -@Deprecated("Use `runTest` instead to support completing from other dispatchers. " + - "Please see the migration guide for details: " + - "https://github.com/Kotlin/kotlinx.coroutines/blob/master/kotlinx-coroutines-test/MIGRATION.md", - level = DeprecationLevel.WARNING) -// Since 1.6.0, ERROR in 1.7.0 and removed as experimental in 1.8.0 +@Deprecated( + "Use `runTest` instead to support completing from other dispatchers. " + + "Please see the migration guide for details: " + + "https://github.com/Kotlin/kotlinx.coroutines/blob/master/kotlinx-coroutines-test/MIGRATION.md", + level = DeprecationLevel.WARNING +) +// Since 1.6.0, kept as warning in 1.7.0, ERROR in 1.8.0 and removed as experimental in 1.9.0 public fun TestCoroutineScope.runBlockingTest(block: suspend TestCoroutineScope.() -> Unit): Unit = runBlockingTest(coroutineContext, block) @@ -129,7 +134,7 @@ public fun TestCoroutineScope.runBlockingTest(block: suspend TestCoroutineScope. * Convenience method for calling [runBlockingTestOnTestScope] on an existing [TestScope]. */ @Deprecated("Use `runTest` instead to support completing from other dispatchers.", level = DeprecationLevel.WARNING) -// Since 1.6.0, ERROR in 1.7.0 and removed as experimental in 1.8.0 +// Since 1.6.0, kept as warning in 1.7.0, ERROR in 1.8.0 and removed as experimental in 1.9.0 public fun TestScope.runBlockingTest(block: suspend TestScope.() -> Unit): Unit = runBlockingTestOnTestScope(coroutineContext, block) @@ -141,11 +146,13 @@ public fun TestScope.runBlockingTest(block: suspend TestScope.() -> Unit): Unit * [migration guide](https://github.com/Kotlin/kotlinx.coroutines/blob/master/kotlinx-coroutines-test/MIGRATION.md) * for an instruction on how to update the code for the new API. */ -@Deprecated("Use `runTest` instead to support completing from other dispatchers. " + - "Please see the migration guide for details: " + - "https://github.com/Kotlin/kotlinx.coroutines/blob/master/kotlinx-coroutines-test/MIGRATION.md", - level = DeprecationLevel.WARNING) -// Since 1.6.0, ERROR in 1.7.0 and removed as experimental in 1.8.0 +@Deprecated( + "Use `runTest` instead to support completing from other dispatchers. " + + "Please see the migration guide for details: " + + "https://github.com/Kotlin/kotlinx.coroutines/blob/master/kotlinx-coroutines-test/MIGRATION.md", + level = DeprecationLevel.WARNING +) +// Since 1.6.0, kept as warning in 1.7.0, ERROR in 1.8.0 and removed as experimental in 1.9.0 public fun TestCoroutineDispatcher.runBlockingTest(block: suspend TestCoroutineScope.() -> Unit): Unit = runBlockingTest(this, block) @@ -154,17 +161,22 @@ public fun TestCoroutineDispatcher.runBlockingTest(block: suspend TestCoroutineS */ @ExperimentalCoroutinesApi @Deprecated("Use `runTest` instead.", level = DeprecationLevel.WARNING) -// Since 1.6.0, ERROR in 1.7.0 and removed as experimental in 1.8.0 +// Since 1.6.0, kept as warning in 1.7.0, ERROR in 1.8.0 and removed as experimental in 1.9.0 public fun runTestWithLegacyScope( context: CoroutineContext = EmptyCoroutineContext, dispatchTimeoutMs: Long = DEFAULT_DISPATCH_TIMEOUT_MS, testBody: suspend TestCoroutineScope.() -> Unit -): TestResult { +) { if (context[RunningInRunTest] != null) throw IllegalStateException("Calls to `runTest` can't be nested. Please read the docs on `TestResult` for details.") val testScope = TestBodyCoroutine(createTestCoroutineScope(context + RunningInRunTest)) return createTestResult { - runTestCoroutine(testScope, dispatchTimeoutMs, TestBodyCoroutine::tryGetCompletionCause, testBody) { + runTestCoroutineLegacy( + testScope, + dispatchTimeoutMs.milliseconds, + TestBodyCoroutine::tryGetCompletionCause, + testBody + ) { try { testScope.cleanup() emptyList() @@ -188,7 +200,7 @@ public fun runTestWithLegacyScope( */ @ExperimentalCoroutinesApi @Deprecated("Use `TestScope.runTest` instead.", level = DeprecationLevel.WARNING) -// Since 1.6.0, ERROR in 1.7.0 and removed as experimental in 1.8.0 +// Since 1.6.0, kept as warning in 1.7.0, ERROR in 1.8.0 and removed as experimental in 1.9.0 public fun TestCoroutineScope.runTest( dispatchTimeoutMs: Long = DEFAULT_DISPATCH_TIMEOUT_MS, block: suspend TestCoroutineScope.() -> Unit diff --git a/kotlinx-coroutines-test/jvm/src/migration/TestCoroutineDispatcher.kt b/kotlinx-coroutines-test/jvm/src/migration/TestCoroutineDispatcher.kt index 08f428f2..3f049b6f 100644 --- a/kotlinx-coroutines-test/jvm/src/migration/TestCoroutineDispatcher.kt +++ b/kotlinx-coroutines-test/jvm/src/migration/TestCoroutineDispatcher.kt @@ -24,7 +24,7 @@ import kotlin.coroutines.* @Deprecated("The execution order of `TestCoroutineDispatcher` can be confusing, and the mechanism of " + "pausing is typically misunderstood. Please use `StandardTestDispatcher` or `UnconfinedTestDispatcher` instead.", level = DeprecationLevel.WARNING) -// Since 1.6.0, ERROR in 1.7.0 and removed as experimental in 1.8.0 +// Since 1.6.0, kept as warning in 1.7.0, ERROR in 1.8.0 and removed as experimental in 1.9.0 public class TestCoroutineDispatcher(public override val scheduler: TestCoroutineScheduler = TestCoroutineScheduler()): TestDispatcher(), Delay, SchedulerAsDelayController { @@ -61,6 +61,10 @@ public class TestCoroutineDispatcher(public override val scheduler: TestCoroutin scheduler.registerEvent(this, 0, block, context) { false } /** @suppress */ + @Deprecated( + "Please use a dispatcher that is paused by default, like `StandardTestDispatcher`.", + level = DeprecationLevel.ERROR + ) override suspend fun pauseDispatcher(block: suspend () -> Unit) { val previous = dispatchImmediately dispatchImmediately = false @@ -72,11 +76,19 @@ public class TestCoroutineDispatcher(public override val scheduler: TestCoroutin } /** @suppress */ + @Deprecated( + "Please use a dispatcher that is paused by default, like `StandardTestDispatcher`.", + level = DeprecationLevel.ERROR + ) override fun pauseDispatcher() { dispatchImmediately = false } /** @suppress */ + @Deprecated( + "Please use a dispatcher that is paused by default, like `StandardTestDispatcher`.", + level = DeprecationLevel.ERROR + ) override fun resumeDispatcher() { dispatchImmediately = true } diff --git a/kotlinx-coroutines-test/jvm/src/migration/TestCoroutineExceptionHandler.kt b/kotlinx-coroutines-test/jvm/src/migration/TestCoroutineExceptionHandler.kt index 9da521f0..150055f5 100644 --- a/kotlinx-coroutines-test/jvm/src/migration/TestCoroutineExceptionHandler.kt +++ b/kotlinx-coroutines-test/jvm/src/migration/TestCoroutineExceptionHandler.kt @@ -16,7 +16,7 @@ import kotlin.coroutines.* "Consider whether the default mechanism of handling uncaught exceptions is sufficient. " + "If not, try writing your own `CoroutineExceptionHandler` and " + "please report your use case at https://github.com/Kotlin/kotlinx.coroutines/issues.", - level = DeprecationLevel.WARNING + level = DeprecationLevel.ERROR ) // Since 1.6.0, ERROR in 1.7.0 and removed as experimental in 1.8.0 public interface UncaughtExceptionCaptor { @@ -42,10 +42,11 @@ public interface UncaughtExceptionCaptor { /** * An exception handler that captures uncaught exceptions in tests. */ +@Suppress("DEPRECATION_ERROR") @Deprecated( "Deprecated for removal without a replacement. " + "It may be to define one's own `CoroutineExceptionHandler` if you just need to handle '" + - "uncaught exceptions without a special `TestCoroutineScope` integration.", level = DeprecationLevel.WARNING + "uncaught exceptions without a special `TestCoroutineScope` integration.", level = DeprecationLevel.ERROR ) // Since 1.6.0, ERROR in 1.7.0 and removed as experimental in 1.8.0 public class TestCoroutineExceptionHandler : @@ -58,7 +59,7 @@ public class TestCoroutineExceptionHandler : override fun handleException(context: CoroutineContext, exception: Throwable) { synchronized(_lock) { if (_coroutinesCleanedUp) { - handleCoroutineExceptionImpl(context, exception) + handleUncaughtCoroutineException(context, exception) } _exceptions += exception } diff --git a/kotlinx-coroutines-test/jvm/src/migration/TestCoroutineScope.kt b/kotlinx-coroutines-test/jvm/src/migration/TestCoroutineScope.kt index 4a2cbc5c..4a503c5e 100644 --- a/kotlinx-coroutines-test/jvm/src/migration/TestCoroutineScope.kt +++ b/kotlinx-coroutines-test/jvm/src/migration/TestCoroutineScope.kt @@ -1,7 +1,7 @@ /* * Copyright 2016-2021 JetBrains s.r.o. Use of this source code is governed by the Apache 2.0 license. */ -@file:Suppress("DEPRECATION") +@file:Suppress("DEPRECATION_ERROR", "DEPRECATION") package kotlinx.coroutines.test @@ -22,7 +22,7 @@ import kotlin.coroutines.* "Please see the migration guide for details: " + "https://github.com/Kotlin/kotlinx.coroutines/blob/master/kotlinx-coroutines-test/MIGRATION.md", level = DeprecationLevel.WARNING) -// Since 1.6.0, ERROR in 1.7.0 and removed as experimental in 1.8.0 +// Since 1.6.0, kept as warning in 1.7.0, ERROR in 1.8.0 and removed as experimental in 1.9.0 public interface TestCoroutineScope : CoroutineScope { /** * Called after the test completes. @@ -45,7 +45,7 @@ public interface TestCoroutineScope : CoroutineScope { */ @ExperimentalCoroutinesApi @Deprecated("Please call `runTest`, which automatically performs the cleanup, instead of using this function.") - // Since 1.6.0, ERROR in 1.7.0 and removed as experimental in 1.8.0 + // Since 1.6.0, kept as warning in 1.7.0, ERROR in 1.8.0 and removed as experimental in 1.9.0 public fun cleanupTestCoroutines() /** @@ -86,6 +86,7 @@ private class TestCoroutineScopeImpl( /** These jobs existed before the coroutine scope was used, so it's alright if they don't get cancelled. */ private val initialJobs = coroutineContext.activeJobs() + @Deprecated("Please call `runTest`, which automatically performs the cleanup, instead of using this function.") override fun cleanupTestCoroutines() { val delayController = coroutineContext.delayController val hasUnfinishedJobs = if (delayController != null) { @@ -138,7 +139,7 @@ internal fun CoroutineContext.activeJobs(): Set<Job> { ), level = DeprecationLevel.WARNING ) -// Since 1.6.0, ERROR in 1.7.0 and removed as experimental in 1.8.0 +// Since 1.6.0, kept as warning in 1.7.0, ERROR in 1.8.0 and removed as experimental in 1.9.0 public fun TestCoroutineScope(context: CoroutineContext = EmptyCoroutineContext): TestCoroutineScope { val scheduler = context[TestCoroutineScheduler] ?: TestCoroutineScheduler() return createTestCoroutineScope(TestCoroutineDispatcher(scheduler) + TestCoroutineExceptionHandler() + context) @@ -180,7 +181,7 @@ public fun TestCoroutineScope(context: CoroutineContext = EmptyCoroutineContext) "Please use TestScope() construction instead, or just runTest(), without creating a scope.", level = DeprecationLevel.WARNING ) -// Since 1.6.0, ERROR in 1.7.0 and removed as experimental in 1.8.0 +// Since 1.6.0, kept as warning in 1.7.0, ERROR in 1.8.0 and removed as experimental in 1.9.0 public fun createTestCoroutineScope(context: CoroutineContext = EmptyCoroutineContext): TestCoroutineScope { val ctxWithDispatcher = context.withDelaySkipping() var scope: TestCoroutineScopeImpl? = null @@ -238,7 +239,7 @@ public val TestCoroutineScope.currentTime: Long "The name of this function is misleading: it not only advances the time, but also runs the tasks " + "scheduled *at* the ending moment.", ReplaceWith("this.testScheduler.apply { advanceTimeBy(delayTimeMillis); runCurrent() }"), - DeprecationLevel.WARNING + DeprecationLevel.ERROR ) // Since 1.6.0, ERROR in 1.7.0 and removed as experimental in 1.8.0 public fun TestCoroutineScope.advanceTimeBy(delayTimeMillis: Long): Unit = @@ -282,7 +283,7 @@ public fun TestCoroutineScope.runCurrent() { "(this.coroutineContext[ContinuationInterceptor]!! as DelayController).pauseDispatcher(block)", "kotlin.coroutines.ContinuationInterceptor" ), - DeprecationLevel.WARNING + DeprecationLevel.ERROR ) // Since 1.6.0, ERROR in 1.7.0 and removed as experimental in 1.8.0 public suspend fun TestCoroutineScope.pauseDispatcher(block: suspend () -> Unit) { @@ -298,7 +299,7 @@ public suspend fun TestCoroutineScope.pauseDispatcher(block: suspend () -> Unit) "(this.coroutineContext[ContinuationInterceptor]!! as DelayController).pauseDispatcher()", "kotlin.coroutines.ContinuationInterceptor" ), - level = DeprecationLevel.WARNING + level = DeprecationLevel.ERROR ) // Since 1.6.0, ERROR in 1.7.0 and removed as experimental in 1.8.0 public fun TestCoroutineScope.pauseDispatcher() { @@ -314,7 +315,7 @@ public fun TestCoroutineScope.pauseDispatcher() { "(this.coroutineContext[ContinuationInterceptor]!! as DelayController).resumeDispatcher()", "kotlin.coroutines.ContinuationInterceptor" ), - level = DeprecationLevel.WARNING + level = DeprecationLevel.ERROR ) // Since 1.6.0, ERROR in 1.7.0 and removed as experimental in 1.8.0 public fun TestCoroutineScope.resumeDispatcher() { @@ -334,8 +335,9 @@ public fun TestCoroutineScope.resumeDispatcher() { "easily misused. It is only present for backward compatibility and will be removed in the subsequent " + "releases. If you need to check the list of exceptions, please consider creating your own " + "`CoroutineExceptionHandler`.", - level = DeprecationLevel.WARNING + level = DeprecationLevel.ERROR ) +// Since 1.6.0, ERROR in 1.7.0 and removed as experimental in 1.8.0 public val TestCoroutineScope.uncaughtExceptions: List<Throwable> get() = (coroutineContext[CoroutineExceptionHandler] as? UncaughtExceptionCaptor)?.uncaughtExceptions ?: emptyList() diff --git a/kotlinx-coroutines-test/jvm/src/module-info.java b/kotlinx-coroutines-test/jvm/src/module-info.java new file mode 100644 index 00000000..9846263c --- /dev/null +++ b/kotlinx-coroutines-test/jvm/src/module-info.java @@ -0,0 +1,15 @@ +import kotlinx.coroutines.CoroutineExceptionHandler; +import kotlinx.coroutines.internal.MainDispatcherFactory; +import kotlinx.coroutines.test.internal.ExceptionCollectorAsService; +import kotlinx.coroutines.test.internal.TestMainDispatcherFactory; + +module kotlinx.coroutines.test { + requires kotlin.stdlib; + requires kotlinx.coroutines.core; + requires kotlinx.atomicfu; + + exports kotlinx.coroutines.test; + + provides MainDispatcherFactory with TestMainDispatcherFactory; + provides CoroutineExceptionHandler with ExceptionCollectorAsService; +} diff --git a/kotlinx-coroutines-test/jvm/test/DumpOnTimeoutTest.kt b/kotlinx-coroutines-test/jvm/test/DumpOnTimeoutTest.kt new file mode 100644 index 00000000..814e5f06 --- /dev/null +++ b/kotlinx-coroutines-test/jvm/test/DumpOnTimeoutTest.kt @@ -0,0 +1,48 @@ +/* + * Copyright 2016-2023 JetBrains s.r.o. Use of this source code is governed by the Apache 2.0 license. + */ + +package kotlinx.coroutines.test + +import kotlinx.coroutines.* +import kotlinx.coroutines.debug.* +import org.junit.Test +import java.io.* +import kotlin.test.* +import kotlin.time.Duration.Companion.milliseconds + +class DumpOnTimeoutTest { + /** + * Tests that the dump on timeout contains the correct stacktrace. + */ + @Test + fun testDumpOnTimeout() { + val oldErr = System.err + val baos = ByteArrayOutputStream() + try { + System.setErr(PrintStream(baos, true)) + DebugProbes.withDebugProbes { + try { + runTest(timeout = 100.milliseconds) { + uniquelyNamedFunction() + } + throw IllegalStateException("unreachable") + } catch (e: UncompletedCoroutinesError) { + // do nothing + } + } + baos.toString().let { + assertTrue(it.contains("uniquelyNamedFunction"), "Actual trace:\n$it") + } + } finally { + System.setErr(oldErr) + } + } + + fun CoroutineScope.uniquelyNamedFunction() { + while (true) { + ensureActive() + Thread.sleep(10) + } + } +} diff --git a/kotlinx-coroutines-test/jvm/test/HelpersJvm.kt b/kotlinx-coroutines-test/jvm/test/HelpersJvm.kt index e9aa3ff7..8d40b078 100644 --- a/kotlinx-coroutines-test/jvm/test/HelpersJvm.kt +++ b/kotlinx-coroutines-test/jvm/test/HelpersJvm.kt @@ -3,8 +3,11 @@ */ package kotlinx.coroutines.test -actual fun testResultMap(block: (() -> Unit) -> Unit, test: () -> TestResult) { - block { - test() +actual fun testResultChain(block: () -> TestResult, after: (Result<Unit>) -> TestResult): TestResult { + try { + block() + after(Result.success(Unit)) + } catch (e: Throwable) { + after(Result.failure(e)) } } diff --git a/kotlinx-coroutines-test/jvm/test/MemoryLeakTest.kt b/kotlinx-coroutines-test/jvm/test/MemoryLeakTest.kt new file mode 100644 index 00000000..705c97ea --- /dev/null +++ b/kotlinx-coroutines-test/jvm/test/MemoryLeakTest.kt @@ -0,0 +1,23 @@ +/* + * Copyright 2016-2022 JetBrains s.r.o. Use of this source code is governed by the Apache 2.0 license. + */ +import kotlinx.coroutines.* +import kotlinx.coroutines.test.* +import kotlin.test.* + +class MemoryLeakTest { + + @Test + fun testCancellationLeakInTestCoroutineScheduler() = runTest { + val leakingObject = Any() + val job = launch(start = CoroutineStart.UNDISPATCHED) { + delay(1) + // This code is needed to hold a reference to `leakingObject` until the job itself is weakly reachable. + leakingObject.hashCode() + } + job.cancel() + FieldWalker.assertReachableCount(1, testScheduler) { it === leakingObject } + runCurrent() + FieldWalker.assertReachableCount(0, testScheduler) { it === leakingObject } + } +} diff --git a/kotlinx-coroutines-test/jvm/test/MultithreadingTest.kt b/kotlinx-coroutines-test/jvm/test/MultithreadingTest.kt index 90a16d06..2ac577c4 100644 --- a/kotlinx-coroutines-test/jvm/test/MultithreadingTest.kt +++ b/kotlinx-coroutines-test/jvm/test/MultithreadingTest.kt @@ -99,14 +99,24 @@ class MultithreadingTest { } } - /** Tests that [StandardTestDispatcher] is confined to the thread that interacts with the scheduler. */ + /** Tests that [StandardTestDispatcher] is not executed in-place but confined to the thread in which the + * virtual time control happens. */ @Test - fun testStandardTestDispatcherIsConfined() = runTest { + fun testStandardTestDispatcherIsConfined(): Unit = runBlocking { + val scheduler = TestCoroutineScheduler() val initialThread = Thread.currentThread() - withContext(Dispatchers.IO) { - val ioThread = Thread.currentThread() - assertNotSame(initialThread, ioThread) + val job = launch(StandardTestDispatcher(scheduler)) { + assertEquals(initialThread, Thread.currentThread()) + withContext(Dispatchers.IO) { + val ioThread = Thread.currentThread() + assertNotSame(initialThread, ioThread) + } + assertEquals(initialThread, Thread.currentThread()) + } + scheduler.advanceUntilIdle() + while (job.isActive) { + scheduler.receiveDispatchEvent() + scheduler.advanceUntilIdle() } - assertEquals(initialThread, Thread.currentThread()) } } diff --git a/kotlinx-coroutines-test/jvm/test/migration/RunTestLegacyScopeTest.kt b/kotlinx-coroutines-test/jvm/test/migration/RunTestLegacyScopeTest.kt index 7f1dd009..ed5b1577 100644 --- a/kotlinx-coroutines-test/jvm/test/migration/RunTestLegacyScopeTest.kt +++ b/kotlinx-coroutines-test/jvm/test/migration/RunTestLegacyScopeTest.kt @@ -67,19 +67,6 @@ class RunTestLegacyScopeTest { } @Test - fun testRunTestWithZeroTimeoutWithUncontrolledDispatches() = testResultMap({ fn -> - assertFailsWith<UncompletedCoroutinesError> { fn() } - }) { - runTestWithLegacyScope(dispatchTimeoutMs = 0) { - withContext(Dispatchers.Default) { - delay(10) - 3 - } - fail("shouldn't be reached") - } - } - - @Test fun testRunTestWithSmallTimeout() = testResultMap({ fn -> assertFailsWith<UncompletedCoroutinesError> { fn() } }) { diff --git a/kotlinx-coroutines-test/jvm/test/migration/TestBuildersTest.kt b/kotlinx-coroutines-test/jvm/test/migration/TestBuildersTest.kt index 6d49a01f..4b0428d0 100644 --- a/kotlinx-coroutines-test/jvm/test/migration/TestBuildersTest.kt +++ b/kotlinx-coroutines-test/jvm/test/migration/TestBuildersTest.kt @@ -8,7 +8,7 @@ import kotlinx.coroutines.* import kotlin.coroutines.* import kotlin.test.* -@Suppress("DEPRECATION") +@Suppress("DEPRECATION", "DEPRECATION_ERROR") class TestBuildersTest { @Test @@ -129,4 +129,4 @@ class TestBuildersTest { assertEquals(4, calls) } -}
\ No newline at end of file +} diff --git a/kotlinx-coroutines-test/jvm/test/migration/TestCoroutineDispatcherOrderTest.kt b/kotlinx-coroutines-test/jvm/test/migration/TestCoroutineDispatcherOrderTest.kt index 93fcd909..115c2729 100644 --- a/kotlinx-coroutines-test/jvm/test/migration/TestCoroutineDispatcherOrderTest.kt +++ b/kotlinx-coroutines-test/jvm/test/migration/TestCoroutineDispatcherOrderTest.kt @@ -8,7 +8,7 @@ import kotlinx.atomicfu.* import kotlinx.coroutines.* import kotlin.test.* -@Suppress("DEPRECATION") +@Suppress("DEPRECATION", "DEPRECATION_ERROR") class TestCoroutineDispatcherOrderTest: OrderedExecutionTestBase() { @Test @@ -40,4 +40,4 @@ class TestCoroutineDispatcherOrderTest: OrderedExecutionTestBase() { scope.cleanupTestCoroutines() finish(9) } -}
\ No newline at end of file +} diff --git a/kotlinx-coroutines-test/jvm/test/migration/TestCoroutineDispatcherTest.kt b/kotlinx-coroutines-test/jvm/test/migration/TestCoroutineDispatcherTest.kt index a78d923d..ea9762ff 100644 --- a/kotlinx-coroutines-test/jvm/test/migration/TestCoroutineDispatcherTest.kt +++ b/kotlinx-coroutines-test/jvm/test/migration/TestCoroutineDispatcherTest.kt @@ -7,7 +7,7 @@ package kotlinx.coroutines.test import kotlinx.coroutines.* import kotlin.test.* -@Suppress("DEPRECATION") +@Suppress("DEPRECATION", "DEPRECATION_ERROR") class TestCoroutineDispatcherTest { @Test fun whenDispatcherPaused_doesNotAutoProgressCurrent() { @@ -74,4 +74,4 @@ class TestCoroutineDispatcherTest { assertFailsWith<UncompletedCoroutinesError> { subject.cleanupTestCoroutines() } } -}
\ No newline at end of file +} diff --git a/kotlinx-coroutines-test/jvm/test/migration/TestCoroutineExceptionHandlerTest.kt b/kotlinx-coroutines-test/jvm/test/migration/TestCoroutineExceptionHandlerTest.kt index 20da1307..332634ea 100644 --- a/kotlinx-coroutines-test/jvm/test/migration/TestCoroutineExceptionHandlerTest.kt +++ b/kotlinx-coroutines-test/jvm/test/migration/TestCoroutineExceptionHandlerTest.kt @@ -6,7 +6,7 @@ package kotlinx.coroutines.test import kotlin.test.* -@Suppress("DEPRECATION") +@Suppress("DEPRECATION_ERROR") class TestCoroutineExceptionHandlerTest { @Test fun whenExceptionsCaught_availableViaProperty() { @@ -15,4 +15,4 @@ class TestCoroutineExceptionHandlerTest { subject.handleException(subject, expected) assertEquals(listOf(expected), subject.uncaughtExceptions) } -}
\ No newline at end of file +} diff --git a/kotlinx-coroutines-test/jvm/test/migration/TestRunBlockingTest.kt b/kotlinx-coroutines-test/jvm/test/migration/TestRunBlockingTest.kt index af3b2489..ebdd973b 100644 --- a/kotlinx-coroutines-test/jvm/test/migration/TestRunBlockingTest.kt +++ b/kotlinx-coroutines-test/jvm/test/migration/TestRunBlockingTest.kt @@ -7,7 +7,7 @@ package kotlinx.coroutines.test import kotlinx.coroutines.* import kotlin.test.* -@Suppress("DEPRECATION") +@Suppress("DEPRECATION", "DEPRECATION_ERROR") class TestRunBlockingTest { @Test @@ -437,4 +437,4 @@ class TestRunBlockingTest { } } } -}
\ No newline at end of file +} diff --git a/kotlinx-coroutines-test/native/src/TestBuilders.kt b/kotlinx-coroutines-test/native/src/TestBuilders.kt index a9599019..607dec6a 100644 --- a/kotlinx-coroutines-test/native/src/TestBuilders.kt +++ b/kotlinx-coroutines-test/native/src/TestBuilders.kt @@ -4,6 +4,7 @@ package kotlinx.coroutines.test import kotlinx.coroutines.* +import kotlin.native.concurrent.* @Suppress("ACTUAL_WITHOUT_EXPECT") public actual typealias TestResult = Unit @@ -13,3 +14,5 @@ internal actual fun createTestResult(testProcedure: suspend CoroutineScope.() -> testProcedure() } } + +internal actual fun dumpCoroutines() { } diff --git a/kotlinx-coroutines-test/native/test/FailingTests.kt b/kotlinx-coroutines-test/native/test/FailingTests.kt deleted file mode 100644 index 9fb77ce7..00000000 --- a/kotlinx-coroutines-test/native/test/FailingTests.kt +++ /dev/null @@ -1,25 +0,0 @@ -/* - * Copyright 2016-2021 JetBrains s.r.o. Use of this source code is governed by the Apache 2.0 license. - */ - -package kotlinx.coroutines.test - -import kotlinx.coroutines.* -import kotlin.test.* - -/** These are tests that we want to fail. They are here so that, when the issue is fixed, their failure indicates that - * everything is better now. */ -class FailingTests { - @Test - fun testRunTestLoopShutdownOnTimeout() = testResultMap({ fn -> - assertFailsWith<IllegalStateException> { fn() } - }) { - runTest(dispatchTimeoutMs = 1) { - withContext(Dispatchers.Default) { - delay(10000) - } - fail("shouldn't be reached") - } - } - -}
\ No newline at end of file diff --git a/kotlinx-coroutines-test/native/test/Helpers.kt b/kotlinx-coroutines-test/native/test/Helpers.kt index ef478b7e..be615fb0 100644 --- a/kotlinx-coroutines-test/native/test/Helpers.kt +++ b/kotlinx-coroutines-test/native/test/Helpers.kt @@ -5,9 +5,12 @@ package kotlinx.coroutines.test import kotlin.test.* -actual fun testResultMap(block: (() -> Unit) -> Unit, test: () -> TestResult) { - block { - test() +actual fun testResultChain(block: () -> TestResult, after: (Result<Unit>) -> TestResult): TestResult { + try { + block() + after(Result.success(Unit)) + } catch (e: Throwable) { + after(Result.failure(e)) } } diff --git a/license/third_party/minima_LICENSE.txt b/license/third_party/minima_LICENSE.txt deleted file mode 100644 index e8c3c2d5..00000000 --- a/license/third_party/minima_LICENSE.txt +++ /dev/null @@ -1,21 +0,0 @@ -The MIT License (MIT) - -Copyright (c) 2016 Parker Moore - -Permission is hereby granted, free of charge, to any person obtaining a copy -of this software and associated documentation files (the "Software"), to deal -in the Software without restriction, including without limitation the rights -to use, copy, modify, merge, publish, distribute, sublicense, and/or sell -copies of the Software, and to permit persons to whom the Software is -furnished to do so, subject to the following conditions: - -The above copyright notice and this permission notice shall be included in -all copies or substantial portions of the Software. - -THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR -IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, -FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE -AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER -LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, -OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN -THE SOFTWARE. diff --git a/reactive/kotlinx-coroutines-jdk9/build.gradle.kts b/reactive/kotlinx-coroutines-jdk9/build.gradle.kts index be5eb421..0853a34d 100644 --- a/reactive/kotlinx-coroutines-jdk9/build.gradle.kts +++ b/reactive/kotlinx-coroutines-jdk9/build.gradle.kts @@ -6,6 +6,11 @@ dependencies { implementation(project(":kotlinx-coroutines-reactive")) } +java { + sourceCompatibility = JavaVersion.VERSION_1_9 + targetCompatibility = JavaVersion.VERSION_1_9 +} + tasks { compileKotlin { kotlinOptions.jvmTarget = "9" diff --git a/reactive/kotlinx-coroutines-jdk9/src/module-info.java b/reactive/kotlinx-coroutines-jdk9/src/module-info.java new file mode 100644 index 00000000..ce31d810 --- /dev/null +++ b/reactive/kotlinx-coroutines-jdk9/src/module-info.java @@ -0,0 +1,9 @@ +@SuppressWarnings("JavaModuleNaming") +module kotlinx.coroutines.jdk9 { + requires kotlin.stdlib; + requires kotlinx.coroutines.core; + requires kotlinx.coroutines.reactive; + requires org.reactivestreams; + + exports kotlinx.coroutines.jdk9; +} diff --git a/reactive/kotlinx-coroutines-reactive/api/kotlinx-coroutines-reactive.api b/reactive/kotlinx-coroutines-reactive/api/kotlinx-coroutines-reactive.api index b52df185..3a2ea12d 100644 --- a/reactive/kotlinx-coroutines-reactive/api/kotlinx-coroutines-reactive.api +++ b/reactive/kotlinx-coroutines-reactive/api/kotlinx-coroutines-reactive.api @@ -5,9 +5,9 @@ public final class kotlinx/coroutines/reactive/AwaitKt { public static final fun awaitFirstOrNull (Lorg/reactivestreams/Publisher;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; public static final fun awaitLast (Lorg/reactivestreams/Publisher;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; public static final fun awaitSingle (Lorg/reactivestreams/Publisher;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; - public static final fun awaitSingleOrDefault (Lorg/reactivestreams/Publisher;Ljava/lang/Object;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; - public static final fun awaitSingleOrElse (Lorg/reactivestreams/Publisher;Lkotlin/jvm/functions/Function0;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; - public static final fun awaitSingleOrNull (Lorg/reactivestreams/Publisher;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; + public static final synthetic fun awaitSingleOrDefault (Lorg/reactivestreams/Publisher;Ljava/lang/Object;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; + public static final synthetic fun awaitSingleOrElse (Lorg/reactivestreams/Publisher;Lkotlin/jvm/functions/Function0;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; + public static final synthetic fun awaitSingleOrNull (Lorg/reactivestreams/Publisher;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; } public final class kotlinx/coroutines/reactive/ChannelKt { @@ -49,7 +49,7 @@ public final class kotlinx/coroutines/reactive/PublishKt { public static final fun publishInternal (Lkotlinx/coroutines/CoroutineScope;Lkotlin/coroutines/CoroutineContext;Lkotlin/jvm/functions/Function2;Lkotlin/jvm/functions/Function2;)Lorg/reactivestreams/Publisher; } -public final class kotlinx/coroutines/reactive/PublisherCoroutine : kotlinx/coroutines/AbstractCoroutine, kotlinx/coroutines/channels/ProducerScope, kotlinx/coroutines/selects/SelectClause2, org/reactivestreams/Subscription { +public final class kotlinx/coroutines/reactive/PublisherCoroutine : kotlinx/coroutines/AbstractCoroutine, kotlinx/coroutines/channels/ProducerScope, org/reactivestreams/Subscription { public fun <init> (Lkotlin/coroutines/CoroutineContext;Lorg/reactivestreams/Subscriber;Lkotlin/jvm/functions/Function2;)V public fun cancel ()V public fun close (Ljava/lang/Throwable;)Z @@ -60,7 +60,6 @@ public final class kotlinx/coroutines/reactive/PublisherCoroutine : kotlinx/coro public fun isClosedForSend ()Z public fun offer (Ljava/lang/Object;)Z public synthetic fun onCompleted (Ljava/lang/Object;)V - public fun registerSelectClause2 (Lkotlinx/coroutines/selects/SelectInstance;Ljava/lang/Object;Lkotlin/jvm/functions/Function2;)V public fun request (J)V public fun send (Ljava/lang/Object;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; public fun trySend-JP2dKIU (Ljava/lang/Object;)Ljava/lang/Object; diff --git a/reactive/kotlinx-coroutines-reactive/build.gradle.kts b/reactive/kotlinx-coroutines-reactive/build.gradle.kts index c2e4b5c9..fbcde965 100644 --- a/reactive/kotlinx-coroutines-reactive/build.gradle.kts +++ b/reactive/kotlinx-coroutines-reactive/build.gradle.kts @@ -1,7 +1,14 @@ +import kotlinx.kover.gradle.plugin.dsl.* + /* * Copyright 2016-2021 JetBrains s.r.o. Use of this source code is governed by the Apache 2.0 license. */ +plugins { + // apply plugin to use autocomplete for Kover DSL + id("org.jetbrains.kotlinx.kover") +} + val reactiveStreamsVersion = property("reactive_streams_version") dependencies { @@ -35,16 +42,14 @@ externalDocumentationLink( url = "https://www.reactive-streams.org/reactive-streams-$reactiveStreamsVersion-javadoc/" ) -val commonKoverExcludes = listOf( - "kotlinx.coroutines.reactive.FlowKt", // Deprecated - "kotlinx.coroutines.reactive.FlowKt__MigrationKt", // Deprecated - "kotlinx.coroutines.reactive.ConvertKt" // Deprecated -) - -tasks.koverHtmlReport { - excludes = commonKoverExcludes -} - -tasks.koverVerify { - excludes = commonKoverExcludes +koverReport { + filters { + excludes { + classes( + "kotlinx.coroutines.reactive.FlowKt", // Deprecated + "kotlinx.coroutines.reactive.FlowKt__MigrationKt", // Deprecated + "kotlinx.coroutines.reactive.ConvertKt" // Deprecated + ) + } + } } diff --git a/reactive/kotlinx-coroutines-reactive/src/Await.kt b/reactive/kotlinx-coroutines-reactive/src/Await.kt index 3d9a0f85..446d986c 100644 --- a/reactive/kotlinx-coroutines-reactive/src/Await.kt +++ b/reactive/kotlinx-coroutines-reactive/src/Await.kt @@ -106,7 +106,7 @@ public suspend fun <T> Publisher<T>.awaitSingle(): T = awaitOne(Mode.SINGLE) @Deprecated( message = "Deprecated without a replacement due to its name incorrectly conveying the behavior. " + "Please consider using awaitFirstOrDefault().", - level = DeprecationLevel.ERROR + level = DeprecationLevel.HIDDEN ) // Warning since 1.5, error in 1.6, hidden in 1.7 public suspend fun <T> Publisher<T>.awaitSingleOrDefault(default: T): T = awaitOne(Mode.SINGLE_OR_DEFAULT, default) @@ -135,7 +135,7 @@ public suspend fun <T> Publisher<T>.awaitSingleOrDefault(default: T): T = awaitO message = "Deprecated without a replacement due to its name incorrectly conveying the behavior. " + "There is a specialized version for Reactor's Mono, please use that where applicable. " + "Alternatively, please consider using awaitFirstOrNull().", - level = DeprecationLevel.ERROR, + level = DeprecationLevel.HIDDEN, replaceWith = ReplaceWith("this.awaitSingleOrNull()", "kotlinx.coroutines.reactor") ) // Warning since 1.5, error in 1.6, hidden in 1.7 public suspend fun <T> Publisher<T>.awaitSingleOrNull(): T? = awaitOne(Mode.SINGLE_OR_DEFAULT) @@ -164,7 +164,7 @@ public suspend fun <T> Publisher<T>.awaitSingleOrNull(): T? = awaitOne(Mode.SING @Deprecated( message = "Deprecated without a replacement due to its name incorrectly conveying the behavior. " + "Please consider using awaitFirstOrElse().", - level = DeprecationLevel.ERROR + level = DeprecationLevel.HIDDEN ) // Warning since 1.5, error in 1.6, hidden in 1.7 public suspend fun <T> Publisher<T>.awaitSingleOrElse(defaultValue: () -> T): T = awaitOne(Mode.SINGLE_OR_DEFAULT) ?: defaultValue() diff --git a/reactive/kotlinx-coroutines-reactive/src/Channel.kt b/reactive/kotlinx-coroutines-reactive/src/Channel.kt index a8db2171..7836ed7d 100644 --- a/reactive/kotlinx-coroutines-reactive/src/Channel.kt +++ b/reactive/kotlinx-coroutines-reactive/src/Channel.kt @@ -7,7 +7,6 @@ package kotlinx.coroutines.reactive import kotlinx.atomicfu.* import kotlinx.coroutines.channels.* import kotlinx.coroutines.flow.* -import kotlinx.coroutines.internal.* import org.reactivestreams.* /** @@ -29,7 +28,7 @@ internal fun <T> Publisher<T>.toChannel(request: Int = 1): ReceiveChannel<T> { @Suppress("INVISIBLE_REFERENCE", "INVISIBLE_MEMBER", "SubscriberImplementation") private class SubscriptionChannel<T>( private val request: Int -) : LinkedListChannel<T>(null), Subscriber<T> { +) : BufferedChannel<T>(capacity = Channel.UNLIMITED), Subscriber<T> { init { require(request >= 0) { "Invalid request size: $request" } } @@ -40,7 +39,7 @@ private class SubscriptionChannel<T>( // can be negative if we have receivers, but no subscription yet private val _requested = atomic(0) - // --------------------- AbstractChannel overrides ------------------------------- + // --------------------- BufferedChannel overrides ------------------------------- @Suppress("CANNOT_OVERRIDE_INVISIBLE_MEMBER") override fun onReceiveEnqueued() { _requested.loop { wasRequested -> @@ -64,7 +63,7 @@ private class SubscriptionChannel<T>( } @Suppress("CANNOT_OVERRIDE_INVISIBLE_MEMBER") - override fun onClosedIdempotent(closed: LockFreeLinkedListNode) { + override fun onClosedIdempotent() { _subscription.getAndSet(null)?.cancel() // cancel exactly once } diff --git a/reactive/kotlinx-coroutines-reactive/src/Migration.kt b/reactive/kotlinx-coroutines-reactive/src/Migration.kt index 41927e67..858ab00e 100644 --- a/reactive/kotlinx-coroutines-reactive/src/Migration.kt +++ b/reactive/kotlinx-coroutines-reactive/src/Migration.kt @@ -30,7 +30,6 @@ public fun <T : Any> Publisher<T>.asFlowDeprecated(): Flow<T> = asFlow() public fun <T : Any> Flow<T>.asPublisherDeprecated(): Publisher<T> = asPublisher() /** @suppress */ -@FlowPreview @Deprecated( message = "batchSize parameter is deprecated, use .buffer() instead to control the backpressure", level = DeprecationLevel.HIDDEN, diff --git a/reactive/kotlinx-coroutines-reactive/src/Publish.kt b/reactive/kotlinx-coroutines-reactive/src/Publish.kt index 1b8683ce..ae85e418 100644 --- a/reactive/kotlinx-coroutines-reactive/src/Publish.kt +++ b/reactive/kotlinx-coroutines-reactive/src/Publish.kt @@ -6,7 +6,6 @@ package kotlinx.coroutines.reactive import kotlinx.atomicfu.* import kotlinx.coroutines.* import kotlinx.coroutines.channels.* -import kotlinx.coroutines.intrinsics.* import kotlinx.coroutines.selects.* import kotlinx.coroutines.sync.* import org.reactivestreams.* @@ -69,11 +68,9 @@ public class PublisherCoroutine<in T>( parentContext: CoroutineContext, private val subscriber: Subscriber<T>, private val exceptionOnCancelHandler: (Throwable, CoroutineContext) -> Unit -) : AbstractCoroutine<Unit>(parentContext, false, true), ProducerScope<T>, Subscription, SelectClause2<T, SendChannel<T>> { +) : AbstractCoroutine<Unit>(parentContext, false, true), ProducerScope<T>, Subscription { override val channel: SendChannel<T> get() = this - // Mutex is locked when either nRequested == 0 or while subscriber.onXXX is being invoked - private val mutex = Mutex(locked = true) private val _nRequested = atomic(0L) // < 0 when closed (CLOSED or SIGNALLED) @Volatile @@ -84,6 +81,42 @@ public class PublisherCoroutine<in T>( override fun invokeOnClose(handler: (Throwable?) -> Unit): Nothing = throw UnsupportedOperationException("PublisherCoroutine doesn't support invokeOnClose") + // Mutex is locked when either nRequested == 0 or while subscriber.onXXX is being invoked + private val mutex: Mutex = Mutex(locked = true) + + @Suppress("UNCHECKED_CAST", "INVISIBLE_MEMBER") + override val onSend: SelectClause2<T, SendChannel<T>> get() = SelectClause2Impl( + clauseObject = this, + regFunc = PublisherCoroutine<*>::registerSelectForSend as RegistrationFunction, + processResFunc = PublisherCoroutine<*>::processResultSelectSend as ProcessResultFunction + ) + + @Suppress("UNCHECKED_CAST", "UNUSED_PARAMETER") + private fun registerSelectForSend(select: SelectInstance<*>, element: Any?) { + // Try to acquire the mutex and complete in the registration phase. + if (mutex.tryLock()) { + select.selectInRegistrationPhase(Unit) + return + } + // Start a new coroutine that waits for the mutex, invoking `trySelect(..)` after that. + // Please note that at the point of the `trySelect(..)` invocation the corresponding + // `select` can still be in the registration phase, making this `trySelect(..)` bound to fail. + // In this case, the `onSend` clause will be re-registered, which alongside with the mutex + // manipulation makes the resulting solution obstruction-free. + launch { + mutex.lock() + if (!select.trySelect(this@PublisherCoroutine, Unit)) { + mutex.unlock() + } + } + } + + @Suppress("RedundantNullableReturnType", "UNUSED_PARAMETER", "UNCHECKED_CAST") + private fun processResultSelectSend(element: Any?, selectResult: Any?): Any? { + doLockedNext(element as T)?.let { throw it } + return this@PublisherCoroutine + } + override fun trySend(element: T): ChannelResult<Unit> = if (!mutex.tryLock()) { ChannelResult.failure() @@ -99,29 +132,6 @@ public class PublisherCoroutine<in T>( doLockedNext(element)?.let { throw it } } - override val onSend: SelectClause2<T, SendChannel<T>> - get() = this - - // registerSelectSend - @Suppress("PARAMETER_NAME_CHANGED_ON_OVERRIDE") - override fun <R> registerSelectClause2(select: SelectInstance<R>, element: T, block: suspend (SendChannel<T>) -> R) { - val clause = suspend { - doLockedNext(element)?.let { throw it } - block(this) - } - - launch(start = CoroutineStart.UNDISPATCHED) { - mutex.lock() - // Already selected -- bail out - if (!select.trySelect()) { - mutex.unlock() - return@launch - } - - clause.startCoroutineCancellable(select.completion) - } - } - /* * This code is not trivial because of the following properties: * 1. It ensures conformance to the reactive specification that mandates that onXXX invocations should not @@ -214,7 +224,7 @@ public class PublisherCoroutine<in T>( * We have to recheck `isCompleted` after `unlock` anyway. */ mutex.unlock() - // check isCompleted and and try to regain lock to signal completion + // check isCompleted and try to regain lock to signal completion if (isCompleted && mutex.tryLock()) { doLockedSignalCompleted(completionCause, completionCauseHandled) } diff --git a/reactive/kotlinx-coroutines-reactive/src/module-info.java b/reactive/kotlinx-coroutines-reactive/src/module-info.java new file mode 100644 index 00000000..67fcc26b --- /dev/null +++ b/reactive/kotlinx-coroutines-reactive/src/module-info.java @@ -0,0 +1,10 @@ +module kotlinx.coroutines.reactive { + requires kotlin.stdlib; + requires kotlinx.coroutines.core; + requires kotlinx.atomicfu; + requires org.reactivestreams; + + exports kotlinx.coroutines.reactive; + + uses kotlinx.coroutines.reactive.ContextInjector; +} diff --git a/reactive/kotlinx-coroutines-reactor/api/kotlinx-coroutines-reactor.api b/reactive/kotlinx-coroutines-reactor/api/kotlinx-coroutines-reactor.api index 4589117c..5a881a12 100644 --- a/reactive/kotlinx-coroutines-reactor/api/kotlinx-coroutines-reactor.api +++ b/reactive/kotlinx-coroutines-reactor/api/kotlinx-coroutines-reactor.api @@ -1,5 +1,5 @@ public final class kotlinx/coroutines/reactor/ConvertKt { - public static final fun asFlux (Lkotlinx/coroutines/channels/ReceiveChannel;Lkotlin/coroutines/CoroutineContext;)Lreactor/core/publisher/Flux; + public static final synthetic fun asFlux (Lkotlinx/coroutines/channels/ReceiveChannel;Lkotlin/coroutines/CoroutineContext;)Lreactor/core/publisher/Flux; public static synthetic fun asFlux$default (Lkotlinx/coroutines/channels/ReceiveChannel;Lkotlin/coroutines/CoroutineContext;ILjava/lang/Object;)Lreactor/core/publisher/Flux; public static final fun asMono (Lkotlinx/coroutines/Deferred;Lkotlin/coroutines/CoroutineContext;)Lreactor/core/publisher/Mono; public static final fun asMono (Lkotlinx/coroutines/Job;Lkotlin/coroutines/CoroutineContext;)Lreactor/core/publisher/Mono; @@ -17,11 +17,11 @@ public final class kotlinx/coroutines/reactor/FluxKt { } public final class kotlinx/coroutines/reactor/MonoKt { - public static final fun awaitFirst (Lreactor/core/publisher/Mono;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; - public static final fun awaitFirstOrDefault (Lreactor/core/publisher/Mono;Ljava/lang/Object;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; - public static final fun awaitFirstOrElse (Lreactor/core/publisher/Mono;Lkotlin/jvm/functions/Function0;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; - public static final fun awaitFirstOrNull (Lreactor/core/publisher/Mono;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; - public static final fun awaitLast (Lreactor/core/publisher/Mono;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; + public static final synthetic fun awaitFirst (Lreactor/core/publisher/Mono;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; + public static final synthetic fun awaitFirstOrDefault (Lreactor/core/publisher/Mono;Ljava/lang/Object;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; + public static final synthetic fun awaitFirstOrElse (Lreactor/core/publisher/Mono;Lkotlin/jvm/functions/Function0;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; + public static final synthetic fun awaitFirstOrNull (Lreactor/core/publisher/Mono;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; + public static final synthetic fun awaitLast (Lreactor/core/publisher/Mono;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; public static final fun awaitSingle (Lreactor/core/publisher/Mono;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; public static final fun awaitSingleOrNull (Lreactor/core/publisher/Mono;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; public static final fun mono (Lkotlin/coroutines/CoroutineContext;Lkotlin/jvm/functions/Function2;)Lreactor/core/publisher/Mono; diff --git a/reactive/kotlinx-coroutines-reactor/build.gradle.kts b/reactive/kotlinx-coroutines-reactor/build.gradle.kts index d4bb135f..7d3441ca 100644 --- a/reactive/kotlinx-coroutines-reactor/build.gradle.kts +++ b/reactive/kotlinx-coroutines-reactor/build.gradle.kts @@ -2,6 +2,11 @@ * Copyright 2016-2021 JetBrains s.r.o. Use of this source code is governed by the Apache 2.0 license. */ +plugins { + // apply plugin to use autocomplete for Kover DSL + id("org.jetbrains.kotlinx.kover") +} + val reactorVersion = version("reactor") dependencies { @@ -28,15 +33,14 @@ externalDocumentationLink( url = "https://projectreactor.io/docs/core/$reactorVersion/api/" ) -val commonKoverExcludes = listOf( - "kotlinx.coroutines.reactor.FlowKt", // Deprecated - "kotlinx.coroutines.reactor.ConvertKt\$asFlux$1" // Deprecated -) - -tasks.koverHtmlReport { - excludes = commonKoverExcludes -} -tasks.koverVerify { - excludes = commonKoverExcludes +koverReport { + filters { + excludes { + classes( + "kotlinx.coroutines.reactor.FlowKt", // Deprecated + "kotlinx.coroutines.reactor.ConvertKt\$asFlux$1" // Deprecated + ) + } + } } diff --git a/reactive/kotlinx-coroutines-reactor/src/Convert.kt b/reactive/kotlinx-coroutines-reactor/src/Convert.kt index 3063d1dd..efdedf78 100644 --- a/reactive/kotlinx-coroutines-reactor/src/Convert.kt +++ b/reactive/kotlinx-coroutines-reactor/src/Convert.kt @@ -45,7 +45,7 @@ public fun <T> Deferred<T?>.asMono(context: CoroutineContext): Mono<T> = mono(co * @suppress */ @Deprecated(message = "Deprecated in the favour of consumeAsFlow()", - level = DeprecationLevel.ERROR, + level = DeprecationLevel.HIDDEN, replaceWith = ReplaceWith("this.consumeAsFlow().asFlux(context)", imports = ["kotlinx.coroutines.flow.consumeAsFlow"])) public fun <T> ReceiveChannel<T>.asFlux(context: CoroutineContext = EmptyCoroutineContext): Flux<T> = flux(context) { for (t in this@asFlux) diff --git a/reactive/kotlinx-coroutines-reactor/src/Mono.kt b/reactive/kotlinx-coroutines-reactor/src/Mono.kt index f31004b6..27dea603 100644 --- a/reactive/kotlinx-coroutines-reactor/src/Mono.kt +++ b/reactive/kotlinx-coroutines-reactor/src/Mono.kt @@ -45,7 +45,7 @@ public fun <T> mono( */ public suspend fun <T> Mono<T>.awaitSingleOrNull(): T? = suspendCancellableCoroutine { cont -> injectCoroutineContext(cont.context).subscribe(object : Subscriber<T> { - private var seenValue = false + private var value: T? = null override fun onSubscribe(s: Subscription) { cont.invokeOnCancellation { s.cancel() } @@ -53,12 +53,14 @@ public suspend fun <T> Mono<T>.awaitSingleOrNull(): T? = suspendCancellableCorou } override fun onComplete() { - if (!seenValue) cont.resume(null) + cont.resume(value) + value = null } override fun onNext(t: T) { - seenValue = true - cont.resume(t) + // We don't return the value immediately because the process that emitted it may not be finished yet. + // Resuming now could lead to race conditions between emitter and the awaiting code. + value = t } override fun onError(error: Throwable) { cont.resumeWithException(error) } @@ -157,7 +159,7 @@ public fun <T> CoroutineScope.mono( @Deprecated( message = "Mono produces at most one value, so the semantics of dropping the remaining elements are not useful. " + "Please use awaitSingle() instead.", - level = DeprecationLevel.ERROR, + level = DeprecationLevel.HIDDEN, replaceWith = ReplaceWith("this.awaitSingle()") ) // Warning since 1.5, error in 1.6 public suspend fun <T> Mono<T>.awaitFirst(): T = awaitSingle() @@ -181,7 +183,7 @@ public suspend fun <T> Mono<T>.awaitFirst(): T = awaitSingle() @Deprecated( message = "Mono produces at most one value, so the semantics of dropping the remaining elements are not useful. " + "Please use awaitSingleOrNull() instead.", - level = DeprecationLevel.ERROR, + level = DeprecationLevel.HIDDEN, replaceWith = ReplaceWith("this.awaitSingleOrNull() ?: default") ) // Warning since 1.5, error in 1.6 public suspend fun <T> Mono<T>.awaitFirstOrDefault(default: T): T = awaitSingleOrNull() ?: default @@ -205,7 +207,7 @@ public suspend fun <T> Mono<T>.awaitFirstOrDefault(default: T): T = awaitSingleO @Deprecated( message = "Mono produces at most one value, so the semantics of dropping the remaining elements are not useful. " + "Please use awaitSingleOrNull() instead.", - level = DeprecationLevel.ERROR, + level = DeprecationLevel.HIDDEN, replaceWith = ReplaceWith("this.awaitSingleOrNull()") ) // Warning since 1.5, error in 1.6 public suspend fun <T> Mono<T>.awaitFirstOrNull(): T? = awaitSingleOrNull() @@ -229,7 +231,7 @@ public suspend fun <T> Mono<T>.awaitFirstOrNull(): T? = awaitSingleOrNull() @Deprecated( message = "Mono produces at most one value, so the semantics of dropping the remaining elements are not useful. " + "Please use awaitSingleOrNull() instead.", - level = DeprecationLevel.ERROR, + level = DeprecationLevel.HIDDEN, replaceWith = ReplaceWith("this.awaitSingleOrNull() ?: defaultValue()") ) // Warning since 1.5, error in 1.6 public suspend fun <T> Mono<T>.awaitFirstOrElse(defaultValue: () -> T): T = awaitSingleOrNull() ?: defaultValue() @@ -253,7 +255,7 @@ public suspend fun <T> Mono<T>.awaitFirstOrElse(defaultValue: () -> T): T = awai @Deprecated( message = "Mono produces at most one value, so the last element is the same as the first. " + "Please use awaitSingle() instead.", - level = DeprecationLevel.ERROR, + level = DeprecationLevel.HIDDEN, replaceWith = ReplaceWith("this.awaitSingle()") ) // Warning since 1.5, error in 1.6 public suspend fun <T> Mono<T>.awaitLast(): T = awaitSingle() diff --git a/reactive/kotlinx-coroutines-reactor/src/ReactorFlow.kt b/reactive/kotlinx-coroutines-reactor/src/ReactorFlow.kt index 0fc743f9..6a77bbf3 100644 --- a/reactive/kotlinx-coroutines-reactor/src/ReactorFlow.kt +++ b/reactive/kotlinx-coroutines-reactor/src/ReactorFlow.kt @@ -32,8 +32,7 @@ private class FlowAsFlux<T : Any>( private val flow: Flow<T>, private val context: CoroutineContext ) : Flux<T>() { - override fun subscribe(subscriber: CoreSubscriber<in T>?) { - if (subscriber == null) throw NullPointerException() + override fun subscribe(subscriber: CoreSubscriber<in T>) { val hasContext = !subscriber.currentContext().isEmpty val source = if (hasContext) flow.flowOn(subscriber.currentContext().asCoroutineContext()) else flow subscriber.onSubscribe(FlowSubscription(source, subscriber, context)) diff --git a/reactive/kotlinx-coroutines-reactor/src/module-info.java b/reactive/kotlinx-coroutines-reactor/src/module-info.java new file mode 100644 index 00000000..b75308b5 --- /dev/null +++ b/reactive/kotlinx-coroutines-reactor/src/module-info.java @@ -0,0 +1,14 @@ +import kotlinx.coroutines.reactive.ContextInjector; +import kotlinx.coroutines.reactor.ReactorContextInjector; + +module kotlinx.coroutines.reactor { + requires kotlin.stdlib; + requires kotlinx.coroutines.core; + requires kotlinx.coroutines.reactive; + requires org.reactivestreams; + requires reactor.core; + + exports kotlinx.coroutines.reactor; + + provides ContextInjector with ReactorContextInjector; +} diff --git a/reactive/kotlinx-coroutines-reactor/test/MonoAwaitStressTest.kt b/reactive/kotlinx-coroutines-reactor/test/MonoAwaitStressTest.kt new file mode 100644 index 00000000..355aa686 --- /dev/null +++ b/reactive/kotlinx-coroutines-reactor/test/MonoAwaitStressTest.kt @@ -0,0 +1,54 @@ +/* + * Copyright 2016-2022 JetBrains s.r.o. Use of this source code is governed by the Apache 2.0 license. + */ + +package kotlinx.coroutines.reactor + +import kotlinx.coroutines.* +import org.junit.Test +import org.reactivestreams.* +import reactor.core.* +import reactor.core.publisher.* +import kotlin.concurrent.* +import kotlin.test.* + +class MonoAwaitStressTest: TestBase() { + private val N_REPEATS = 10_000 * stressTestMultiplier + + private var completed: Boolean = false + + private var thread: Thread? = null + + /** + * Tests that [Mono.awaitSingleOrNull] does await [CoreSubscriber.onComplete] and does not return + * the value as soon as it has it. + */ + @Test + fun testAwaitingRacingWithCompletion() = runTest { + val mono = object: Mono<Int>() { + override fun subscribe(s: CoreSubscriber<in Int>) { + s.onSubscribe(object : Subscription { + override fun request(n: Long) { + thread = thread { + s.onNext(1) + Thread.yield() + completed = true + s.onComplete() + } + } + + override fun cancel() { + } + }) + } + } + repeat(N_REPEATS) { + thread = null + completed = false + val value = mono.awaitSingleOrNull() + assertTrue(completed, "iteration $it") + assertEquals(1, value) + thread!!.join() + } + } +} diff --git a/reactive/kotlinx-coroutines-rx2/api/kotlinx-coroutines-rx2.api b/reactive/kotlinx-coroutines-rx2/api/kotlinx-coroutines-rx2.api index c2d1c4bf..803ac905 100644 --- a/reactive/kotlinx-coroutines-rx2/api/kotlinx-coroutines-rx2.api +++ b/reactive/kotlinx-coroutines-rx2/api/kotlinx-coroutines-rx2.api @@ -1,13 +1,13 @@ public final class kotlinx/coroutines/rx2/RxAwaitKt { public static final fun await (Lio/reactivex/CompletableSource;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; - public static final fun await (Lio/reactivex/MaybeSource;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; + public static final synthetic fun await (Lio/reactivex/MaybeSource;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; public static final fun await (Lio/reactivex/SingleSource;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; public static final fun awaitFirst (Lio/reactivex/ObservableSource;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; public static final fun awaitFirstOrDefault (Lio/reactivex/ObservableSource;Ljava/lang/Object;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; public static final fun awaitFirstOrElse (Lio/reactivex/ObservableSource;Lkotlin/jvm/functions/Function0;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; public static final fun awaitFirstOrNull (Lio/reactivex/ObservableSource;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; public static final fun awaitLast (Lio/reactivex/ObservableSource;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; - public static final fun awaitOrDefault (Lio/reactivex/MaybeSource;Ljava/lang/Object;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; + public static final synthetic fun awaitOrDefault (Lio/reactivex/MaybeSource;Ljava/lang/Object;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; public static final fun awaitSingle (Lio/reactivex/MaybeSource;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; public static final fun awaitSingle (Lio/reactivex/ObservableSource;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; public static final fun awaitSingleOrNull (Lio/reactivex/MaybeSource;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; @@ -35,7 +35,7 @@ public final class kotlinx/coroutines/rx2/RxConvertKt { public static final fun asFlowable (Lkotlinx/coroutines/flow/Flow;Lkotlin/coroutines/CoroutineContext;)Lio/reactivex/Flowable; public static synthetic fun asFlowable$default (Lkotlinx/coroutines/flow/Flow;Lkotlin/coroutines/CoroutineContext;ILjava/lang/Object;)Lio/reactivex/Flowable; public static final fun asMaybe (Lkotlinx/coroutines/Deferred;Lkotlin/coroutines/CoroutineContext;)Lio/reactivex/Maybe; - public static final fun asObservable (Lkotlinx/coroutines/channels/ReceiveChannel;Lkotlin/coroutines/CoroutineContext;)Lio/reactivex/Observable; + public static final synthetic fun asObservable (Lkotlinx/coroutines/channels/ReceiveChannel;Lkotlin/coroutines/CoroutineContext;)Lio/reactivex/Observable; public static final fun asObservable (Lkotlinx/coroutines/flow/Flow;Lkotlin/coroutines/CoroutineContext;)Lio/reactivex/Observable; public static synthetic fun asObservable$default (Lkotlinx/coroutines/flow/Flow;Lkotlin/coroutines/CoroutineContext;ILjava/lang/Object;)Lio/reactivex/Observable; public static final fun asSingle (Lkotlinx/coroutines/Deferred;Lkotlin/coroutines/CoroutineContext;)Lio/reactivex/Single; diff --git a/reactive/kotlinx-coroutines-rx2/src/RxAwait.kt b/reactive/kotlinx-coroutines-rx2/src/RxAwait.kt index da9809c9..aeaa1f4b 100644 --- a/reactive/kotlinx-coroutines-rx2/src/RxAwait.kt +++ b/reactive/kotlinx-coroutines-rx2/src/RxAwait.kt @@ -24,9 +24,17 @@ import kotlin.coroutines.* */ public suspend fun CompletableSource.await(): Unit = suspendCancellableCoroutine { cont -> subscribe(object : CompletableObserver { - override fun onSubscribe(d: Disposable) { cont.disposeOnCancellation(d) } - override fun onComplete() { cont.resume(Unit) } - override fun onError(e: Throwable) { cont.resumeWithException(e) } + override fun onSubscribe(d: Disposable) { + cont.disposeOnCancellation(d) + } + + override fun onComplete() { + cont.resume(Unit) + } + + override fun onError(e: Throwable) { + cont.resumeWithException(e) + } }) } @@ -41,13 +49,23 @@ public suspend fun CompletableSource.await(): Unit = suspendCancellableCoroutine * If the [Job] of the current coroutine is cancelled or completed while this suspending function is waiting, this * function immediately resumes with [CancellationException] and disposes of its subscription. */ -@Suppress("UNCHECKED_CAST") public suspend fun <T> MaybeSource<T>.awaitSingleOrNull(): T? = suspendCancellableCoroutine { cont -> subscribe(object : MaybeObserver<T> { - override fun onSubscribe(d: Disposable) { cont.disposeOnCancellation(d) } - override fun onComplete() { cont.resume(null) } - override fun onSuccess(t: T) { cont.resume(t) } - override fun onError(error: Throwable) { cont.resumeWithException(error) } + override fun onSubscribe(d: Disposable) { + cont.disposeOnCancellation(d) + } + + override fun onComplete() { + cont.resume(null) + } + + override fun onSuccess(t: T & Any) { + cont.resume(t) + } + + override fun onError(error: Throwable) { + cont.resumeWithException(error) + } }) } @@ -80,7 +98,7 @@ public suspend fun <T> MaybeSource<T>.awaitSingle(): T = awaitSingleOrNull() ?: */ @Deprecated( message = "Deprecated in favor of awaitSingleOrNull()", - level = DeprecationLevel.ERROR, + level = DeprecationLevel.HIDDEN, replaceWith = ReplaceWith("this.awaitSingleOrNull()") ) // Warning since 1.5, error in 1.6, hidden in 1.7 public suspend fun <T> MaybeSource<T>.await(): T? = awaitSingleOrNull() @@ -102,7 +120,7 @@ public suspend fun <T> MaybeSource<T>.await(): T? = awaitSingleOrNull() */ @Deprecated( message = "Deprecated in favor of awaitSingleOrNull()", - level = DeprecationLevel.ERROR, + level = DeprecationLevel.HIDDEN, replaceWith = ReplaceWith("this.awaitSingleOrNull() ?: default") ) // Warning since 1.5, error in 1.6, hidden in 1.7 public suspend fun <T> MaybeSource<T>.awaitOrDefault(default: T): T = awaitSingleOrNull() ?: default @@ -119,9 +137,17 @@ public suspend fun <T> MaybeSource<T>.awaitOrDefault(default: T): T = awaitSingl */ public suspend fun <T> SingleSource<T>.await(): T = suspendCancellableCoroutine { cont -> subscribe(object : SingleObserver<T> { - override fun onSubscribe(d: Disposable) { cont.disposeOnCancellation(d) } - override fun onSuccess(t: T) { cont.resume(t) } - override fun onError(error: Throwable) { cont.resumeWithException(error) } + override fun onSubscribe(d: Disposable) { + cont.disposeOnCancellation(d) + } + + override fun onSuccess(t: T & Any) { + cont.resume(t) + } + + override fun onError(error: Throwable) { + cont.resumeWithException(error) + } }) } @@ -225,7 +251,7 @@ private suspend fun <T> ObservableSource<T>.awaitOne( cont.invokeOnCancellation { sub.dispose() } } - override fun onNext(t: T) { + override fun onNext(t: T & Any) { when (mode) { Mode.FIRST, Mode.FIRST_OR_DEFAULT -> { if (!seenValue) { diff --git a/reactive/kotlinx-coroutines-rx2/src/RxChannel.kt b/reactive/kotlinx-coroutines-rx2/src/RxChannel.kt index fc09bf9e..94c9c222 100644 --- a/reactive/kotlinx-coroutines-rx2/src/RxChannel.kt +++ b/reactive/kotlinx-coroutines-rx2/src/RxChannel.kt @@ -8,7 +8,6 @@ import io.reactivex.* import io.reactivex.disposables.* import kotlinx.atomicfu.* import kotlinx.coroutines.channels.* -import kotlinx.coroutines.internal.* import kotlinx.coroutines.flow.* import kotlinx.coroutines.reactive.* @@ -46,12 +45,12 @@ internal fun <T> ObservableSource<T>.toChannel(): ReceiveChannel<T> { @Suppress("INVISIBLE_REFERENCE", "INVISIBLE_MEMBER") private class SubscriptionChannel<T> : - LinkedListChannel<T>(null), Observer<T>, MaybeObserver<T> + BufferedChannel<T>(capacity = Channel.UNLIMITED), Observer<T>, MaybeObserver<T> { private val _subscription = atomic<Disposable?>(null) @Suppress("CANNOT_OVERRIDE_INVISIBLE_MEMBER") - override fun onClosedIdempotent(closed: LockFreeLinkedListNode) { + override fun onClosedIdempotent() { _subscription.getAndSet(null)?.dispose() // dispose exactly once } @@ -60,12 +59,12 @@ private class SubscriptionChannel<T> : _subscription.value = sub } - override fun onSuccess(t: T) { + override fun onSuccess(t: T & Any) { trySend(t) close(cause = null) } - override fun onNext(t: T) { + override fun onNext(t: T & Any) { trySend(t) // Safe to ignore return value here, expectedly racing with cancellation } @@ -80,7 +79,7 @@ private class SubscriptionChannel<T> : /** @suppress */ @Deprecated(message = "Deprecated in the favour of Flow", level = DeprecationLevel.HIDDEN) // ERROR in 1.4.0, HIDDEN in 1.6.0 -public fun <T> ObservableSource<T>.openSubscription(): ReceiveChannel<T> { +public fun <T> ObservableSource<T & Any>.openSubscription(): ReceiveChannel<T> { val channel = SubscriptionChannel<T>() subscribe(channel) return channel @@ -88,7 +87,7 @@ public fun <T> ObservableSource<T>.openSubscription(): ReceiveChannel<T> { /** @suppress */ @Deprecated(message = "Deprecated in the favour of Flow", level = DeprecationLevel.HIDDEN) // ERROR in 1.4.0, HIDDEN in 1.6.0 -public fun <T> MaybeSource<T>.openSubscription(): ReceiveChannel<T> { +public fun <T> MaybeSource<T & Any>.openSubscription(): ReceiveChannel<T> { val channel = SubscriptionChannel<T>() subscribe(channel) return channel diff --git a/reactive/kotlinx-coroutines-rx2/src/RxConvert.kt b/reactive/kotlinx-coroutines-rx2/src/RxConvert.kt index 497c922c..a92d68e2 100644 --- a/reactive/kotlinx-coroutines-rx2/src/RxConvert.kt +++ b/reactive/kotlinx-coroutines-rx2/src/RxConvert.kt @@ -141,7 +141,7 @@ public fun <T: Any> Flow<T>.asFlowable(context: CoroutineContext = EmptyCoroutin @Deprecated( message = "Deprecated in the favour of Flow", - level = DeprecationLevel.ERROR, + level = DeprecationLevel.HIDDEN, replaceWith = ReplaceWith("this.consumeAsFlow().asObservable(context)", "kotlinx.coroutines.flow.consumeAsFlow") ) // Deprecated since 1.4.0 public fun <T : Any> ReceiveChannel<T>.asObservable(context: CoroutineContext): Observable<T> = rxObservable(context) { diff --git a/reactive/kotlinx-coroutines-rx2/src/RxObservable.kt b/reactive/kotlinx-coroutines-rx2/src/RxObservable.kt index 90e770bb..ad8fac71 100644 --- a/reactive/kotlinx-coroutines-rx2/src/RxObservable.kt +++ b/reactive/kotlinx-coroutines-rx2/src/RxObservable.kt @@ -10,7 +10,6 @@ import kotlinx.atomicfu.* import kotlinx.coroutines.* import kotlinx.coroutines.channels.* import kotlinx.coroutines.internal.* -import kotlinx.coroutines.intrinsics.* import kotlinx.coroutines.selects.* import kotlinx.coroutines.sync.* import kotlin.coroutines.* @@ -58,12 +57,9 @@ private const val SIGNALLED = -2 // already signalled subscriber onCompleted/on private class RxObservableCoroutine<T : Any>( parentContext: CoroutineContext, private val subscriber: ObservableEmitter<T> -) : AbstractCoroutine<Unit>(parentContext, false, true), ProducerScope<T>, SelectClause2<T, SendChannel<T>> { +) : AbstractCoroutine<Unit>(parentContext, false, true), ProducerScope<T> { override val channel: SendChannel<T> get() = this - // Mutex is locked while subscriber.onXXX is being invoked - private val mutex = Mutex() - private val _signal = atomic(OPEN) override val isClosedForSend: Boolean get() = !isActive @@ -71,6 +67,42 @@ private class RxObservableCoroutine<T : Any>( override fun invokeOnClose(handler: (Throwable?) -> Unit) = throw UnsupportedOperationException("RxObservableCoroutine doesn't support invokeOnClose") + // Mutex is locked when either nRequested == 0 or while subscriber.onXXX is being invoked + private val mutex: Mutex = Mutex() + + @Suppress("UNCHECKED_CAST", "INVISIBLE_MEMBER") + override val onSend: SelectClause2<T, SendChannel<T>> get() = SelectClause2Impl( + clauseObject = this, + regFunc = RxObservableCoroutine<*>::registerSelectForSend as RegistrationFunction, + processResFunc = RxObservableCoroutine<*>::processResultSelectSend as ProcessResultFunction + ) + + @Suppress("UNUSED_PARAMETER") + private fun registerSelectForSend(select: SelectInstance<*>, element: Any?) { + // Try to acquire the mutex and complete in the registration phase. + if (mutex.tryLock()) { + select.selectInRegistrationPhase(Unit) + return + } + // Start a new coroutine that waits for the mutex, invoking `trySelect(..)` after that. + // Please note that at the point of the `trySelect(..)` invocation the corresponding + // `select` can still be in the registration phase, making this `trySelect(..)` bound to fail. + // In this case, the `onSend` clause will be re-registered, which alongside with the mutex + // manipulation makes the resulting solution obstruction-free. + launch { + mutex.lock() + if (!select.trySelect(this@RxObservableCoroutine, Unit)) { + mutex.unlock() + } + } + } + + @Suppress("RedundantNullableReturnType", "UNUSED_PARAMETER", "UNCHECKED_CAST") + private fun processResultSelectSend(element: Any?, selectResult: Any?): Any? { + doLockedNext(element as T)?.let { throw it } + return this@RxObservableCoroutine + } + override fun trySend(element: T): ChannelResult<Unit> = if (!mutex.tryLock()) { ChannelResult.failure() @@ -81,39 +113,11 @@ private class RxObservableCoroutine<T : Any>( } } - public override suspend fun send(element: T) { + override suspend fun send(element: T) { mutex.lock() doLockedNext(element)?.let { throw it } } - override val onSend: SelectClause2<T, SendChannel<T>> - get() = this - - // registerSelectSend - @Suppress("PARAMETER_NAME_CHANGED_ON_OVERRIDE") - override fun <R> registerSelectClause2( - select: SelectInstance<R>, - element: T, - block: suspend (SendChannel<T>) -> R - ) { - val clause = suspend { - doLockedNext(element)?.let { throw it } - block(this) - } - - // This is the default replacement proposed in onLock replacement - launch(start = CoroutineStart.UNDISPATCHED) { - mutex.lock() - // Already selected -- bail out - if (!select.trySelect()) { - mutex.unlock() - return@launch - } - - clause.startCoroutineCancellable(select.completion) - } - } - // assert: mutex.isLocked() private fun doLockedNext(elem: T): Throwable? { // check if already closed for send diff --git a/reactive/kotlinx-coroutines-rx2/src/module-info.java b/reactive/kotlinx-coroutines-rx2/src/module-info.java new file mode 100644 index 00000000..539ea3ee --- /dev/null +++ b/reactive/kotlinx-coroutines-rx2/src/module-info.java @@ -0,0 +1,10 @@ +@SuppressWarnings("JavaModuleNaming") +module kotlinx.coroutines.rx2 { + requires kotlin.stdlib; + requires kotlinx.coroutines.core; + requires kotlinx.coroutines.reactive; + requires kotlinx.atomicfu; + requires io.reactivex.rxjava2; + + exports kotlinx.coroutines.rx2; +} diff --git a/reactive/kotlinx-coroutines-rx3/api/kotlinx-coroutines-rx3.api b/reactive/kotlinx-coroutines-rx3/api/kotlinx-coroutines-rx3.api index 5776214b..f86276e1 100644 --- a/reactive/kotlinx-coroutines-rx3/api/kotlinx-coroutines-rx3.api +++ b/reactive/kotlinx-coroutines-rx3/api/kotlinx-coroutines-rx3.api @@ -1,13 +1,13 @@ public final class kotlinx/coroutines/rx3/RxAwaitKt { public static final fun await (Lio/reactivex/rxjava3/core/CompletableSource;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; - public static final fun await (Lio/reactivex/rxjava3/core/MaybeSource;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; + public static final synthetic fun await (Lio/reactivex/rxjava3/core/MaybeSource;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; public static final fun await (Lio/reactivex/rxjava3/core/SingleSource;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; public static final fun awaitFirst (Lio/reactivex/rxjava3/core/ObservableSource;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; public static final fun awaitFirstOrDefault (Lio/reactivex/rxjava3/core/ObservableSource;Ljava/lang/Object;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; public static final fun awaitFirstOrElse (Lio/reactivex/rxjava3/core/ObservableSource;Lkotlin/jvm/functions/Function0;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; public static final fun awaitFirstOrNull (Lio/reactivex/rxjava3/core/ObservableSource;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; public static final fun awaitLast (Lio/reactivex/rxjava3/core/ObservableSource;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; - public static final fun awaitOrDefault (Lio/reactivex/rxjava3/core/MaybeSource;Ljava/lang/Object;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; + public static final synthetic fun awaitOrDefault (Lio/reactivex/rxjava3/core/MaybeSource;Ljava/lang/Object;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; public static final fun awaitSingle (Lio/reactivex/rxjava3/core/MaybeSource;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; public static final fun awaitSingle (Lio/reactivex/rxjava3/core/ObservableSource;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; public static final fun awaitSingleOrNull (Lio/reactivex/rxjava3/core/MaybeSource;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; diff --git a/reactive/kotlinx-coroutines-rx3/build.gradle b/reactive/kotlinx-coroutines-rx3/build.gradle index 15ef66da..7676b6e2 100644 --- a/reactive/kotlinx-coroutines-rx3/build.gradle +++ b/reactive/kotlinx-coroutines-rx3/build.gradle @@ -23,7 +23,7 @@ compileKotlin { tasks.withType(DokkaTaskPartial.class) { dokkaSourceSets.configureEach { externalDocumentationLink { - url.set(new URL('http://reactivex.io/RxJava/3.x/javadoc/')) + url.set(new URL('https://reactivex.io/RxJava/3.x/javadoc/')) packageListUrl.set(projectDir.toPath().resolve("package.list").toUri().toURL()) } } diff --git a/reactive/kotlinx-coroutines-rx3/src/RxAwait.kt b/reactive/kotlinx-coroutines-rx3/src/RxAwait.kt index 754dd794..33ec8488 100644 --- a/reactive/kotlinx-coroutines-rx3/src/RxAwait.kt +++ b/reactive/kotlinx-coroutines-rx3/src/RxAwait.kt @@ -41,12 +41,11 @@ public suspend fun CompletableSource.await(): Unit = suspendCancellableCoroutine * If the [Job] of the current coroutine is cancelled or completed while this suspending function is waiting, this * function immediately resumes with [CancellationException] and disposes of its subscription. */ -@Suppress("UNCHECKED_CAST") -public suspend fun <T> MaybeSource<T>.awaitSingleOrNull(): T? = suspendCancellableCoroutine { cont -> - subscribe(object : MaybeObserver<T> { +public suspend fun <T> MaybeSource<T & Any>.awaitSingleOrNull(): T? = suspendCancellableCoroutine { cont -> + subscribe(object : MaybeObserver<T & Any> { override fun onSubscribe(d: Disposable) { cont.disposeOnCancellation(d) } override fun onComplete() { cont.resume(null) } - override fun onSuccess(t: T) { cont.resume(t) } + override fun onSuccess(t: T & Any) { cont.resume(t) } override fun onError(error: Throwable) { cont.resumeWithException(error) } }) } @@ -61,7 +60,7 @@ public suspend fun <T> MaybeSource<T>.awaitSingleOrNull(): T? = suspendCancellab * * @throws NoSuchElementException if no elements were produced by this [MaybeSource]. */ -public suspend fun <T> MaybeSource<T>.awaitSingle(): T = awaitSingleOrNull() ?: throw NoSuchElementException() +public suspend fun <T> MaybeSource<T & Any>.awaitSingle(): T = awaitSingleOrNull() ?: throw NoSuchElementException() /** * Awaits for completion of the maybe without blocking a thread. @@ -81,10 +80,10 @@ public suspend fun <T> MaybeSource<T>.awaitSingle(): T = awaitSingleOrNull() ?: */ @Deprecated( message = "Deprecated in favor of awaitSingleOrNull()", - level = DeprecationLevel.ERROR, + level = DeprecationLevel.HIDDEN, replaceWith = ReplaceWith("this.awaitSingleOrNull()") ) // Warning since 1.5, error in 1.6, hidden in 1.7 -public suspend fun <T> MaybeSource<T>.await(): T? = awaitSingleOrNull() +public suspend fun <T> MaybeSource<T & Any>.await(): T? = awaitSingleOrNull() /** * Awaits for completion of the maybe without blocking a thread. @@ -104,10 +103,10 @@ public suspend fun <T> MaybeSource<T>.await(): T? = awaitSingleOrNull() */ @Deprecated( message = "Deprecated in favor of awaitSingleOrNull()", - level = DeprecationLevel.ERROR, + level = DeprecationLevel.HIDDEN, replaceWith = ReplaceWith("this.awaitSingleOrNull() ?: default") ) // Warning since 1.5, error in 1.6, hidden in 1.7 -public suspend fun <T> MaybeSource<T>.awaitOrDefault(default: T): T = awaitSingleOrNull() ?: default +public suspend fun <T> MaybeSource<T & Any>.awaitOrDefault(default: T): T = awaitSingleOrNull() ?: default // ------------------------ SingleSource ------------------------ @@ -119,10 +118,10 @@ public suspend fun <T> MaybeSource<T>.awaitOrDefault(default: T): T = awaitSingl * If the [Job] of the current coroutine is cancelled or completed while the suspending function is waiting, this * function immediately disposes of its subscription and resumes with [CancellationException]. */ -public suspend fun <T> SingleSource<T>.await(): T = suspendCancellableCoroutine { cont -> - subscribe(object : SingleObserver<T> { +public suspend fun <T> SingleSource<T & Any>.await(): T = suspendCancellableCoroutine { cont -> + subscribe(object : SingleObserver<T & Any> { override fun onSubscribe(d: Disposable) { cont.disposeOnCancellation(d) } - override fun onSuccess(t: T) { cont.resume(t) } + override fun onSuccess(t: T & Any) { cont.resume(t) } override fun onError(error: Throwable) { cont.resumeWithException(error) } }) } @@ -139,7 +138,8 @@ public suspend fun <T> SingleSource<T>.await(): T = suspendCancellableCoroutine * * @throws NoSuchElementException if the observable does not emit any value */ -public suspend fun <T> ObservableSource<T>.awaitFirst(): T = awaitOne(Mode.FIRST) +@Suppress("UNCHECKED_CAST") +public suspend fun <T> ObservableSource<T & Any>.awaitFirst(): T = awaitOne(Mode.FIRST) as T /** * Awaits the first value from the given [Observable], or returns the [default] value if none is emitted, without @@ -150,7 +150,9 @@ public suspend fun <T> ObservableSource<T>.awaitFirst(): T = awaitOne(Mode.FIRST * If the [Job] of the current coroutine is cancelled or completed while the suspending function is waiting, this * function immediately disposes of its subscription and resumes with [CancellationException]. */ -public suspend fun <T> ObservableSource<T>.awaitFirstOrDefault(default: T): T = awaitOne(Mode.FIRST_OR_DEFAULT, default) +@Suppress("UNCHECKED_CAST") +public suspend fun <T> ObservableSource<T & Any>.awaitFirstOrDefault(default: T): T = + awaitOne(Mode.FIRST_OR_DEFAULT, default) as T /** * Awaits the first value from the given [Observable], or returns `null` if none is emitted, without blocking the @@ -161,7 +163,7 @@ public suspend fun <T> ObservableSource<T>.awaitFirstOrDefault(default: T): T = * If the [Job] of the current coroutine is cancelled or completed while the suspending function is waiting, this * function immediately disposes of its subscription and resumes with [CancellationException]. */ -public suspend fun <T> ObservableSource<T>.awaitFirstOrNull(): T? = awaitOne(Mode.FIRST_OR_DEFAULT) +public suspend fun <T> ObservableSource<T & Any>.awaitFirstOrNull(): T? = awaitOne(Mode.FIRST_OR_DEFAULT) /** * Awaits the first value from the given [Observable], or calls [defaultValue] to get a value if none is emitted, @@ -172,7 +174,7 @@ public suspend fun <T> ObservableSource<T>.awaitFirstOrNull(): T? = awaitOne(Mod * If the [Job] of the current coroutine is cancelled or completed while the suspending function is waiting, this * function immediately disposes of its subscription and resumes with [CancellationException]. */ -public suspend fun <T> ObservableSource<T>.awaitFirstOrElse(defaultValue: () -> T): T = +public suspend fun <T> ObservableSource<T & Any>.awaitFirstOrElse(defaultValue: () -> T): T = awaitOne(Mode.FIRST_OR_DEFAULT) ?: defaultValue() /** @@ -185,7 +187,8 @@ public suspend fun <T> ObservableSource<T>.awaitFirstOrElse(defaultValue: () -> * * @throws NoSuchElementException if the observable does not emit any value */ -public suspend fun <T> ObservableSource<T>.awaitLast(): T = awaitOne(Mode.LAST) +@Suppress("UNCHECKED_CAST") +public suspend fun <T> ObservableSource<T & Any>.awaitLast(): T = awaitOne(Mode.LAST) as T /** * Awaits the single value from the given observable without blocking the thread and returns the resulting value, or, @@ -198,14 +201,15 @@ public suspend fun <T> ObservableSource<T>.awaitLast(): T = awaitOne(Mode.LAST) * @throws NoSuchElementException if the observable does not emit any value * @throws IllegalArgumentException if the observable emits more than one value */ -public suspend fun <T> ObservableSource<T>.awaitSingle(): T = awaitOne(Mode.SINGLE) +@Suppress("UNCHECKED_CAST") +public suspend fun <T> ObservableSource<T & Any>.awaitSingle(): T = awaitOne(Mode.SINGLE) as T // ------------------------ private ------------------------ internal fun CancellableContinuation<*>.disposeOnCancellation(d: Disposable) = invokeOnCancellation { d.dispose() } -private enum class Mode(val s: String) { +private enum class Mode(@JvmField val s: String) { FIRST("awaitFirst"), FIRST_OR_DEFAULT("awaitFirstOrDefault"), LAST("awaitLast"), @@ -213,11 +217,11 @@ private enum class Mode(val s: String) { override fun toString(): String = s } -private suspend fun <T> ObservableSource<T>.awaitOne( +private suspend fun <T> ObservableSource<T & Any>.awaitOne( mode: Mode, default: T? = null -): T = suspendCancellableCoroutine { cont -> - subscribe(object : Observer<T> { +): T? = suspendCancellableCoroutine { cont -> + subscribe(object : Observer<T & Any> { private lateinit var subscription: Disposable private var value: T? = null private var seenValue = false @@ -227,7 +231,7 @@ private suspend fun <T> ObservableSource<T>.awaitOne( cont.invokeOnCancellation { sub.dispose() } } - override fun onNext(t: T) { + override fun onNext(t: T & Any) { when (mode) { Mode.FIRST, Mode.FIRST_OR_DEFAULT -> { if (!seenValue) { diff --git a/reactive/kotlinx-coroutines-rx3/src/RxChannel.kt b/reactive/kotlinx-coroutines-rx3/src/RxChannel.kt index 21238d24..61449443 100644 --- a/reactive/kotlinx-coroutines-rx3/src/RxChannel.kt +++ b/reactive/kotlinx-coroutines-rx3/src/RxChannel.kt @@ -8,7 +8,6 @@ import io.reactivex.rxjava3.core.* import io.reactivex.rxjava3.disposables.* import kotlinx.atomicfu.* import kotlinx.coroutines.channels.* -import kotlinx.coroutines.internal.* import kotlinx.coroutines.flow.* /** @@ -19,7 +18,7 @@ import kotlinx.coroutines.flow.* * [MaybeSource] doesn't have a corresponding [Flow] adapter, so it should be transformed to [Observable] first. */ @PublishedApi -internal fun <T> MaybeSource<T>.openSubscription(): ReceiveChannel<T> { +internal fun <T> MaybeSource<T & Any>.openSubscription(): ReceiveChannel<T> { val channel = SubscriptionChannel<T>() subscribe(channel) return channel @@ -33,7 +32,7 @@ internal fun <T> MaybeSource<T>.openSubscription(): ReceiveChannel<T> { * [ObservableSource] doesn't have a corresponding [Flow] adapter, so it should be transformed to [Observable] first. */ @PublishedApi -internal fun <T> ObservableSource<T>.openSubscription(): ReceiveChannel<T> { +internal fun <T> ObservableSource<T & Any>.openSubscription(): ReceiveChannel<T> { val channel = SubscriptionChannel<T>() subscribe(channel) return channel @@ -45,7 +44,7 @@ internal fun <T> ObservableSource<T>.openSubscription(): ReceiveChannel<T> { * If [action] throws an exception at some point or if the [MaybeSource] raises an error, the exception is rethrown from * [collect]. */ -public suspend inline fun <T> MaybeSource<T>.collect(action: (T) -> Unit): Unit = +public suspend inline fun <T> MaybeSource<T & Any>.collect(action: (T) -> Unit): Unit = openSubscription().consumeEach(action) /** @@ -54,17 +53,16 @@ public suspend inline fun <T> MaybeSource<T>.collect(action: (T) -> Unit): Unit * If [action] throws an exception at some point, the subscription is cancelled, and the exception is rethrown from * [collect]. Also, if the [ObservableSource] signals an error, that error is rethrown from [collect]. */ -public suspend inline fun <T> ObservableSource<T>.collect(action: (T) -> Unit): Unit = - openSubscription().consumeEach(action) +public suspend inline fun <T> ObservableSource<T & Any>.collect(action: (T) -> Unit): Unit = openSubscription().consumeEach(action) @Suppress("INVISIBLE_REFERENCE", "INVISIBLE_MEMBER") private class SubscriptionChannel<T> : - LinkedListChannel<T>(null), Observer<T>, MaybeObserver<T> + BufferedChannel<T>(capacity = Channel.UNLIMITED), Observer<T & Any>, MaybeObserver<T & Any> { private val _subscription = atomic<Disposable?>(null) @Suppress("CANNOT_OVERRIDE_INVISIBLE_MEMBER") - override fun onClosedIdempotent(closed: LockFreeLinkedListNode) { + override fun onClosedIdempotent() { _subscription.getAndSet(null)?.dispose() // dispose exactly once } @@ -73,12 +71,12 @@ private class SubscriptionChannel<T> : _subscription.value = sub } - override fun onSuccess(t: T) { + override fun onSuccess(t: T & Any) { trySend(t) close(cause = null) } - override fun onNext(t: T) { + override fun onNext(t: T & Any) { trySend(t) // Safe to ignore return value here, expectedly racing with cancellation } diff --git a/reactive/kotlinx-coroutines-rx3/src/RxConvert.kt b/reactive/kotlinx-coroutines-rx3/src/RxConvert.kt index b4693a55..57d2dfb3 100644 --- a/reactive/kotlinx-coroutines-rx3/src/RxConvert.kt +++ b/reactive/kotlinx-coroutines-rx3/src/RxConvert.kt @@ -42,7 +42,7 @@ public fun Job.asCompletable(context: CoroutineContext): Completable = rxComplet * * @param context -- the coroutine context from which the resulting maybe is going to be signalled */ -public fun <T> Deferred<T?>.asMaybe(context: CoroutineContext): Maybe<T> = rxMaybe(context) { +public fun <T> Deferred<T?>.asMaybe(context: CoroutineContext): Maybe<T & Any> = rxMaybe(context) { this@asMaybe.await() } diff --git a/reactive/kotlinx-coroutines-rx3/src/RxMaybe.kt b/reactive/kotlinx-coroutines-rx3/src/RxMaybe.kt index 12d0197b..defb2b72 100644 --- a/reactive/kotlinx-coroutines-rx3/src/RxMaybe.kt +++ b/reactive/kotlinx-coroutines-rx3/src/RxMaybe.kt @@ -20,7 +20,7 @@ import kotlin.coroutines.* public fun <T> rxMaybe( context: CoroutineContext = EmptyCoroutineContext, block: suspend CoroutineScope.() -> T? -): Maybe<T> { +): Maybe<T & Any> { require(context[Job] === null) { "Maybe context cannot contain job in it." + "Its lifecycle should be managed via Disposable handle. Had $context" } return rxMaybeInternal(GlobalScope, context, block) @@ -30,18 +30,18 @@ private fun <T> rxMaybeInternal( scope: CoroutineScope, // support for legacy rxMaybe in scope context: CoroutineContext, block: suspend CoroutineScope.() -> T? -): Maybe<T> = Maybe.create { subscriber -> +): Maybe<T & Any> = Maybe.create { subscriber -> val newContext = scope.newCoroutineContext(context) val coroutine = RxMaybeCoroutine(newContext, subscriber) subscriber.setCancellable(RxCancellable(coroutine)) coroutine.start(CoroutineStart.DEFAULT, coroutine, block) } -private class RxMaybeCoroutine<T>( +private class RxMaybeCoroutine<T: Any>( parentContext: CoroutineContext, private val subscriber: MaybeEmitter<T> -) : AbstractCoroutine<T>(parentContext, false, true) { - override fun onCompleted(value: T) { +) : AbstractCoroutine<T?>(parentContext, false, true) { + override fun onCompleted(value: T?) { try { if (value == null) subscriber.onComplete() else subscriber.onSuccess(value) } catch (e: Throwable) { diff --git a/reactive/kotlinx-coroutines-rx3/src/RxObservable.kt b/reactive/kotlinx-coroutines-rx3/src/RxObservable.kt index 1c5f7c0a..8ea761c9 100644 --- a/reactive/kotlinx-coroutines-rx3/src/RxObservable.kt +++ b/reactive/kotlinx-coroutines-rx3/src/RxObservable.kt @@ -13,7 +13,6 @@ import kotlinx.coroutines.selects.* import kotlinx.coroutines.sync.* import kotlin.coroutines.* import kotlinx.coroutines.internal.* -import kotlinx.coroutines.intrinsics.* /** * Creates cold [observable][Observable] that will run a given [block] in a coroutine. @@ -58,12 +57,9 @@ private const val SIGNALLED = -2 // already signalled subscriber onCompleted/on private class RxObservableCoroutine<T : Any>( parentContext: CoroutineContext, private val subscriber: ObservableEmitter<T> -) : AbstractCoroutine<Unit>(parentContext, false, true), ProducerScope<T>, SelectClause2<T, SendChannel<T>> { +) : AbstractCoroutine<Unit>(parentContext, false, true), ProducerScope<T> { override val channel: SendChannel<T> get() = this - // Mutex is locked while subscriber.onXXX is being invoked - private val mutex = Mutex() - private val _signal = atomic(OPEN) override val isClosedForSend: Boolean get() = !isActive @@ -71,6 +67,42 @@ private class RxObservableCoroutine<T : Any>( override fun invokeOnClose(handler: (Throwable?) -> Unit) = throw UnsupportedOperationException("RxObservableCoroutine doesn't support invokeOnClose") + // Mutex is locked when either nRequested == 0 or while subscriber.onXXX is being invoked + private val mutex: Mutex = Mutex() + + @Suppress("UNCHECKED_CAST", "INVISIBLE_MEMBER") + override val onSend: SelectClause2<T, SendChannel<T>> get() = SelectClause2Impl( + clauseObject = this, + regFunc = RxObservableCoroutine<*>::registerSelectForSend as RegistrationFunction, + processResFunc = RxObservableCoroutine<*>::processResultSelectSend as ProcessResultFunction + ) + + @Suppress("UNUSED_PARAMETER") + private fun registerSelectForSend(select: SelectInstance<*>, element: Any?) { + // Try to acquire the mutex and complete in the registration phase. + if (mutex.tryLock()) { + select.selectInRegistrationPhase(Unit) + return + } + // Start a new coroutine that waits for the mutex, invoking `trySelect(..)` after that. + // Please note that at the point of the `trySelect(..)` invocation the corresponding + // `select` can still be in the registration phase, making this `trySelect(..)` bound to fail. + // In this case, the `onSend` clause will be re-registered, which alongside with the mutex + // manipulation makes the resulting solution obstruction-free. + launch { + mutex.lock() + if (!select.trySelect(this@RxObservableCoroutine, Unit)) { + mutex.unlock() + } + } + } + + @Suppress("RedundantNullableReturnType", "UNUSED_PARAMETER", "UNCHECKED_CAST") + private fun processResultSelectSend(element: Any?, selectResult: Any?): Any? { + doLockedNext(element as T)?.let { throw it } + return this@RxObservableCoroutine + } + override fun trySend(element: T): ChannelResult<Unit> = if (!mutex.tryLock()) { ChannelResult.failure() @@ -81,39 +113,11 @@ private class RxObservableCoroutine<T : Any>( } } - public override suspend fun send(element: T) { + override suspend fun send(element: T) { mutex.lock() doLockedNext(element)?.let { throw it } } - override val onSend: SelectClause2<T, SendChannel<T>> - get() = this - - // registerSelectSend - @Suppress("PARAMETER_NAME_CHANGED_ON_OVERRIDE") - override fun <R> registerSelectClause2( - select: SelectInstance<R>, - element: T, - block: suspend (SendChannel<T>) -> R - ) { - val clause = suspend { - doLockedNext(element)?.let { throw it } - block(this) - } - - // This is the default replacement proposed in onLock replacement - launch(start = CoroutineStart.UNDISPATCHED) { - mutex.lock() - // Already selected -- bail out - if (!select.trySelect()) { - mutex.unlock() - return@launch - } - - clause.startCoroutineCancellable(select.completion) - } - } - // assert: mutex.isLocked() private fun doLockedNext(elem: T): Throwable? { // check if already closed for send diff --git a/reactive/kotlinx-coroutines-rx3/src/RxScheduler.kt b/reactive/kotlinx-coroutines-rx3/src/RxScheduler.kt index abaf0245..e7f93868 100644 --- a/reactive/kotlinx-coroutines-rx3/src/RxScheduler.kt +++ b/reactive/kotlinx-coroutines-rx3/src/RxScheduler.kt @@ -9,7 +9,6 @@ import io.reactivex.rxjava3.disposables.* import io.reactivex.rxjava3.plugins.* import kotlinx.atomicfu.* import kotlinx.coroutines.* -import kotlinx.coroutines.CancellationException import kotlinx.coroutines.channels.* import java.util.concurrent.* import kotlin.coroutines.* diff --git a/reactive/kotlinx-coroutines-rx3/src/module-info.java b/reactive/kotlinx-coroutines-rx3/src/module-info.java new file mode 100644 index 00000000..d57d5279 --- /dev/null +++ b/reactive/kotlinx-coroutines-rx3/src/module-info.java @@ -0,0 +1,10 @@ +@SuppressWarnings("JavaModuleNaming") +module kotlinx.coroutines.rx3 { + requires kotlin.stdlib; + requires kotlinx.coroutines.core; + requires kotlinx.coroutines.reactive; + requires kotlinx.atomicfu; + requires io.reactivex.rxjava3; + + exports kotlinx.coroutines.rx3; +} diff --git a/reactive/kotlinx-coroutines-rx3/test/Check.kt b/reactive/kotlinx-coroutines-rx3/test/Check.kt index 3d4704f4..37a1b324 100644 --- a/reactive/kotlinx-coroutines-rx3/test/Check.kt +++ b/reactive/kotlinx-coroutines-rx3/test/Check.kt @@ -7,7 +7,7 @@ package kotlinx.coroutines.rx3 import io.reactivex.rxjava3.core.* import io.reactivex.rxjava3.plugins.* -fun <T> checkSingleValue( +fun <T : Any> checkSingleValue( observable: Observable<T>, checker: (T) -> Unit ) { @@ -16,15 +16,15 @@ fun <T> checkSingleValue( } fun checkErroneous( - observable: Observable<*>, - checker: (Throwable) -> Unit + observable: Observable<*>, + checker: (Throwable) -> Unit ) { val singleNotification = observable.materialize().blockingSingle() val error = singleNotification.error ?: error("Excepted error") checker(error) } -fun <T> checkSingleValue( +fun <T : Any> checkSingleValue( single: Single<T>, checker: (T) -> Unit ) { @@ -45,8 +45,8 @@ fun checkErroneous( } fun <T> checkMaybeValue( - maybe: Maybe<T>, - checker: (T?) -> Unit + maybe: Maybe<T>, + checker: (T?) -> Unit ) { val maybeValue = maybe.toFlowable().blockingIterable().firstOrNull() checker(maybeValue) diff --git a/settings.gradle b/settings.gradle index f0a76489..151c087f 100644 --- a/settings.gradle +++ b/settings.gradle @@ -5,10 +5,7 @@ pluginManagement { plugins { id "org.openjfx.javafxplugin" version javafx_plugin_version - - // JMH - id "net.ltgt.apt" version "0.21" - id "me.champeau.gradle.jmh" version "0.5.3" + id "me.champeau.jmh" version "0.6.8" } repositories { diff --git a/ui/coroutines-guide-ui.md b/ui/coroutines-guide-ui.md index 4ee898e2..3d11254e 100644 --- a/ui/coroutines-guide-ui.md +++ b/ui/coroutines-guide-ui.md @@ -110,7 +110,7 @@ Add dependencies on `kotlinx-coroutines-android` module to the `dependencies { . `app/build.gradle` file: ```groovy -implementation "org.jetbrains.kotlinx:kotlinx-coroutines-android:1.6.4" +implementation "org.jetbrains.kotlinx:kotlinx-coroutines-android:1.7.2" ``` You can clone [kotlinx.coroutines](https://github.com/Kotlin/kotlinx.coroutines) project from GitHub onto your @@ -335,10 +335,12 @@ processed. This is also a desired behaviour for UI applications that have to react to incoming high-frequency event streams by updating their UI based on the most recently received update. A coroutine that is using -`ConflatedChannel` avoids delays that are usually introduced by buffering of events. +a conflated channel (`capacity = Channel.CONFLATED`, or a buffered channel with +`onBufferOverflow = DROP_OLDEST` or `onBufferOverflow = DROP_LATEST`) avoids delays +that are usually introduced by buffering of events. You can experiment with `capacity` parameter in the above line to see how it affects the behaviour of the code. -Setting `capacity = Channel.UNLIMITED` creates a coroutine with `LinkedListChannel` mailbox that buffers all +Setting `capacity = Channel.UNLIMITED` creates a coroutine with an unbounded mailbox that buffers all events. In this case, the animation runs as many times as the circle is clicked. ## Blocking operations diff --git a/ui/kotlinx-coroutines-android/build.gradle.kts b/ui/kotlinx-coroutines-android/build.gradle.kts index 7618c529..70bc6158 100644 --- a/ui/kotlinx-coroutines-android/build.gradle.kts +++ b/ui/kotlinx-coroutines-android/build.gradle.kts @@ -112,9 +112,3 @@ open class RunR8 : JavaExec() { super.exec() } } - -tasks.withType<Test> { - extensions.configure<KoverTaskExtension> { - excludes = excludes + listOf("com.android.*", "android.*") // Exclude robolectric-generated classes - } -} diff --git a/ui/kotlinx-coroutines-android/src/HandlerDispatcher.kt b/ui/kotlinx-coroutines-android/src/HandlerDispatcher.kt index 5e33169d..7012c23e 100644 --- a/ui/kotlinx-coroutines-android/src/HandlerDispatcher.kt +++ b/ui/kotlinx-coroutines-android/src/HandlerDispatcher.kt @@ -185,22 +185,27 @@ private var choreographer: Choreographer? = null public suspend fun awaitFrame(): Long { // fast path when choreographer is already known val choreographer = choreographer - if (choreographer != null) { - return suspendCancellableCoroutine { cont -> + return if (choreographer != null) { + suspendCancellableCoroutine { cont -> postFrameCallback(choreographer, cont) } + } else { + awaitFrameSlowPath() } - // post into looper thread to figure it out - return suspendCancellableCoroutine { cont -> - Dispatchers.Main.dispatch(EmptyCoroutineContext, Runnable { +} + +private suspend fun awaitFrameSlowPath(): Long = suspendCancellableCoroutine { cont -> + if (Looper.myLooper() === Looper.getMainLooper()) { // Check if we are already in the main looper thread + updateChoreographerAndPostFrameCallback(cont) + } else { // post into looper thread to figure it out + Dispatchers.Main.dispatch(cont.context, Runnable { updateChoreographerAndPostFrameCallback(cont) }) } } private fun updateChoreographerAndPostFrameCallback(cont: CancellableContinuation<Long>) { - val choreographer = choreographer ?: - Choreographer.getInstance()!!.also { choreographer = it } + val choreographer = choreographer ?: Choreographer.getInstance()!!.also { choreographer = it } postFrameCallback(choreographer, cont) } diff --git a/ui/kotlinx-coroutines-android/src/module-info.java b/ui/kotlinx-coroutines-android/src/module-info.java new file mode 100644 index 00000000..71984400 --- /dev/null +++ b/ui/kotlinx-coroutines-android/src/module-info.java @@ -0,0 +1,11 @@ +import kotlinx.coroutines.android.AndroidDispatcherFactory; +import kotlinx.coroutines.internal.MainDispatcherFactory; + +module kotlinx.coroutines.android { + requires kotlin.stdlib; + requires kotlinx.coroutines.core; + + exports kotlinx.coroutines.android; + + provides MainDispatcherFactory with AndroidDispatcherFactory; +} diff --git a/ui/kotlinx-coroutines-android/test/HandlerDispatcherTest.kt b/ui/kotlinx-coroutines-android/test/HandlerDispatcherTest.kt index fe97ae8d..afe6cff2 100644 --- a/ui/kotlinx-coroutines-android/test/HandlerDispatcherTest.kt +++ b/ui/kotlinx-coroutines-android/test/HandlerDispatcherTest.kt @@ -109,16 +109,12 @@ class HandlerDispatcherTest : TestBase() { launch(Dispatchers.Main, start = CoroutineStart.UNDISPATCHED) { expect(1) awaitFrame() - expect(5) + expect(3) } expect(2) // Run choreographer detection mainLooper.runOneTask() - expect(3) - mainLooper.scheduler.advanceBy(50, TimeUnit.MILLISECONDS) - expect(4) - mainLooper.scheduler.advanceBy(51, TimeUnit.MILLISECONDS) - finish(6) + finish(4) } private fun CoroutineScope.doTestAwaitWithDetectedChoreographer() { diff --git a/ui/kotlinx-coroutines-javafx/build.gradle.kts b/ui/kotlinx-coroutines-javafx/build.gradle.kts index f9f66249..634423a5 100644 --- a/ui/kotlinx-coroutines-javafx/build.gradle.kts +++ b/ui/kotlinx-coroutines-javafx/build.gradle.kts @@ -2,8 +2,15 @@ * Copyright 2016-2021 JetBrains s.r.o. Use of this source code is governed by the Apache 2.0 license. */ +buildscript { + dependencies { + // this line can be removed when https://github.com/openjfx/javafx-gradle-plugin/pull/135 is released + classpath("org.javamodularity:moduleplugin:1.8.12") + } +} + plugins { - id("org.openjfx.javafxplugin") version "0.0.9" + id("org.openjfx.javafxplugin") version "0.0.13" } configurations { @@ -22,39 +29,20 @@ javafx { configuration = "javafx" } -val JDK_18: String? by lazy { - System.getenv("JDK_18") -} - -val checkJdk8 by tasks.registering { - // only fail w/o JDK_18 when actually trying to test, not during project setup phase - doLast { - if (JDK_18 == null) { - throw GradleException( - """ - JDK_18 environment variable is not defined. - Can't run JDK 8 compatibility tests. - Please ensure JDK 8 is installed and that JDK_18 points to it. - """.trimIndent() - ) +// Fixup moduleplugin in order to properly run with classpath +tasks { + test { + extensions.configure(org.javamodularity.moduleplugin.extensions.TestModuleOptions::class) { + addReads["kotlinx.coroutines.core"] = "junit" + addReads["kotlinx.coroutines.javafx"] = "kotlin.test" } + jvmArgs = listOf( + "--patch-module", + "kotlinx.coroutines.core=${ + project(":kotlinx-coroutines-core").tasks.named<org.jetbrains.kotlin.gradle.tasks.KotlinCompile>( + "compileTestKotlinJvm" + ).get().destinationDirectory.get() + }" + ) } } - -val jdk8Test by tasks.registering(Test::class) { - // Run these tests only during nightly stress test - onlyIf { project.properties["stressTest"] != null } - - val test = tasks.test.get() - - classpath = test.classpath - testClassesDirs = test.testClassesDirs - executable = "$JDK_18/bin/java" - - dependsOn("compileTestKotlin") - dependsOn(checkJdk8) -} - -tasks.build { - dependsOn(jdk8Test) -} diff --git a/ui/kotlinx-coroutines-javafx/src/JavaFxDispatcher.kt b/ui/kotlinx-coroutines-javafx/src/JavaFxDispatcher.kt index d158fb74..61583aad 100644 --- a/ui/kotlinx-coroutines-javafx/src/JavaFxDispatcher.kt +++ b/ui/kotlinx-coroutines-javafx/src/JavaFxDispatcher.kt @@ -34,7 +34,7 @@ public sealed class JavaFxDispatcher : MainCoroutineDispatcher(), Delay { /** @suppress */ override fun scheduleResumeAfterDelay(timeMillis: Long, continuation: CancellableContinuation<Unit>) { - val timeline = schedule(timeMillis, TimeUnit.MILLISECONDS) { + val timeline = schedule(timeMillis) { with(continuation) { resumeUndispatched(Unit) } } continuation.invokeOnCancellation { timeline.stop() } @@ -42,14 +42,14 @@ public sealed class JavaFxDispatcher : MainCoroutineDispatcher(), Delay { /** @suppress */ override fun invokeOnTimeout(timeMillis: Long, block: Runnable, context: CoroutineContext): DisposableHandle { - val timeline = schedule(timeMillis, TimeUnit.MILLISECONDS) { + val timeline = schedule(timeMillis) { block.run() } return DisposableHandle { timeline.stop() } } - private fun schedule(time: Long, unit: TimeUnit, handler: EventHandler<ActionEvent>): Timeline = - Timeline(KeyFrame(Duration.millis(unit.toMillis(time).toDouble()), handler)).apply { play() } + private fun schedule(timeMillis: Long, handler: EventHandler<ActionEvent>): Timeline = + Timeline(KeyFrame(Duration.millis(timeMillis.toDouble()), handler)).apply { play() } } internal class JavaFxDispatcherFactory : MainDispatcherFactory { @@ -97,7 +97,7 @@ public suspend fun awaitPulse(): Long = suspendCancellableCoroutine { cont -> } private class PulseTimer : AnimationTimer() { - val next = CopyOnWriteArrayList<CancellableContinuation<Long>>() + private val next = CopyOnWriteArrayList<CancellableContinuation<Long>>() override fun handle(now: Long) { val cur = next.toTypedArray() @@ -116,6 +116,7 @@ internal fun initPlatform(): Boolean = PlatformInitializer.success // Lazily try to initialize JavaFx platform just once private object PlatformInitializer { + @JvmField val success = run { /* * Try to instantiate JavaFx platform in a way which works diff --git a/ui/kotlinx-coroutines-javafx/src/module-info.java b/ui/kotlinx-coroutines-javafx/src/module-info.java new file mode 100644 index 00000000..d9b47e5e --- /dev/null +++ b/ui/kotlinx-coroutines-javafx/src/module-info.java @@ -0,0 +1,13 @@ +import kotlinx.coroutines.internal.MainDispatcherFactory; +import kotlinx.coroutines.javafx.JavaFxDispatcherFactory; + +module kotlinx.coroutines.javafx { + requires kotlin.stdlib; + requires kotlinx.coroutines.core; + requires javafx.base; + requires javafx.graphics; + + exports kotlinx.coroutines.javafx; + + provides MainDispatcherFactory with JavaFxDispatcherFactory; +} diff --git a/ui/kotlinx-coroutines-swing/src/SwingDispatcher.kt b/ui/kotlinx-coroutines-swing/src/SwingDispatcher.kt index 3b43483d..010f18c6 100644 --- a/ui/kotlinx-coroutines-swing/src/SwingDispatcher.kt +++ b/ui/kotlinx-coroutines-swing/src/SwingDispatcher.kt @@ -7,7 +7,6 @@ package kotlinx.coroutines.swing import kotlinx.coroutines.* import kotlinx.coroutines.internal.* import java.awt.event.* -import java.util.concurrent.* import javax.swing.* import kotlin.coroutines.* @@ -29,26 +28,22 @@ public sealed class SwingDispatcher : MainCoroutineDispatcher(), Delay { /** @suppress */ override fun scheduleResumeAfterDelay(timeMillis: Long, continuation: CancellableContinuation<Unit>) { - val timer = schedule(timeMillis, TimeUnit.MILLISECONDS, ActionListener { + val timer = schedule(timeMillis) { with(continuation) { resumeUndispatched(Unit) } - }) + } continuation.invokeOnCancellation { timer.stop() } } /** @suppress */ override fun invokeOnTimeout(timeMillis: Long, block: Runnable, context: CoroutineContext): DisposableHandle { - val timer = schedule(timeMillis, TimeUnit.MILLISECONDS, ActionListener { + val timer = schedule(timeMillis) { block.run() - }) - return object : DisposableHandle { - override fun dispose() { - timer.stop() - } } + return DisposableHandle { timer.stop() } } - private fun schedule(time: Long, unit: TimeUnit, action: ActionListener): Timer = - Timer(unit.toMillis(time).coerceAtMost(Int.MAX_VALUE.toLong()).toInt(), action).apply { + private fun schedule(timeMillis: Long, action: ActionListener): Timer = + Timer(timeMillis.coerceAtMost(Int.MAX_VALUE.toLong()).toInt(), action).apply { isRepeats = false start() } diff --git a/ui/kotlinx-coroutines-swing/src/module-info.java b/ui/kotlinx-coroutines-swing/src/module-info.java new file mode 100644 index 00000000..62744873 --- /dev/null +++ b/ui/kotlinx-coroutines-swing/src/module-info.java @@ -0,0 +1,12 @@ +import kotlinx.coroutines.internal.MainDispatcherFactory; +import kotlinx.coroutines.swing.SwingDispatcherFactory; + +module kotlinx.coroutines.swing { + requires kotlin.stdlib; + requires kotlinx.coroutines.core; + requires java.desktop; + + exports kotlinx.coroutines.swing; + + provides MainDispatcherFactory with SwingDispatcherFactory; +} |
