Add wasm branch scheme to update-checkout-config #1

MaxDesiatov · 2019-04-29T20:58:14Z

This allows cloning all of the repositories directly with ./swift/utils/update-checkout --clone --scheme wasm without relying on paths hardcoded in swiftwasm-sdk. It also clones icu that way as well, as that one is used when building on Linux. Later we could add "WebAssembly" platform to update-checkout-config.json to clone it only for WebAssembly and Linux platforms, but for now it's pulled for all platforms to make things easy.

zhuowei · 2019-04-29T21:15:47Z

@MaxDesiatov Thanks!

I haven't set up a branch scheme in update checkout since I'm hoping to upstream most of this - this shouldn't impact that, right?

For CI, I'm using submodules to make sure I check out the exact commits that are know to work

zhuowei · 2019-04-29T21:23:22Z

Merged - thanks!

MaxDesiatov · 2019-04-29T21:25:16Z

@zhuowei Thanks for having a look! My understanding was that the main swiftwasm is the work in progress branch and upstream patches would be in separate branches. So the reasoning was to have the scheme to ease the development process.

This allows cloning all of the repositories directly with ./swift/utils/update-checkout --clone --scheme wasm without relying on paths hardcoded in swiftwasm-sdk. It also clones icu that way as well, as that one is used when building on Linux. Later we could add "WebAssembly" platform to update-checkout-config.json to clone it only for WebAssembly and Linux platforms, but for now it's pulled for all platforms to make things easy. #1

This allows cloning all of the repositories directly with ./swift/utils/update-checkout --clone --scheme wasm without relying on paths hardcoded in swiftwasm-sdk. It also clones icu that way as well, as that one is used when building on Linux. Later we could add "WebAssembly" platform to update-checkout-config.json to clone it only for WebAssembly and Linux platforms, but for now it's pulled for all platforms to make things easy. swiftwasm#1

@inline

The implementation was done quite a while ago. Now, that we have support in lldb (swiftlang/llvm-project#773), we can enable it by default in the compiler. LLDB now shows the runtime failure reason, for example: * thread #1, queue = 'com.apple.main-thread', stop reason = Swift runtime failure: arithmetic overflow frame #1: 0x0000000100000f0d a.out`testit(a=127) at trap_message.swift:4 1 2 @inline(never) 3 func testit(_ a: Int8) -> Int8 { -> 4 return a + 1 5 } 6 For details, see swiftlang#25978 rdar://problem/51278690

- Properly clone and use debug scopes for all instructions in pullback functions. - Emit `debug_value` instructions for adjoint values. - Add debug locations and variable info to adjoint buffer allocations. - Add `TangentBuilder` (a `SILBuilder` subclass) to unify and simplify special emitter utilities for tangent vector code generation. More simplifications to come. Pullback variable inspection example: ```console (lldb) n Process 50984 stopped * thread #1, queue = 'com.apple.main-thread', stop reason = step over frame #0: 0x0000000100003497 main`pullback of foo(x=0) at main.swift:12:11 9 import _Differentiation 10 11 func foo(_ x: Float) -> Float { -> 12 let y = sin(x) 13 let z = cos(y) 14 let k = tanh(z) + cos(y) 15 return k Target 0: (main) stopped. (lldb) fr v (Float) x = 0 (Float) k = 1 (Float) z = 0.495846391 (Float) y = -0.689988375 ``` Resolves rdar://68616528 / SR-13535.

…est1 [CodeCompletion] Migrate some tests to batch completion test #1

* Synchronize both versions of actor_counters.swift test * Synchronize on Job address Make sure to synchronize on Job address (AsyncTasks are Jobs, but not all Jobs are AsyncTasks). * Add fprintf debug output for TSan acquire/release * Add tsan_release edge on task creation without this, we are getting false data races between when a task is created and immediately scheduled on a different thread. False positive for `Sanitizers/tsan/actor_counters.swift` test: ``` WARNING: ThreadSanitizer: data race (pid=81452) Read of size 8 at 0x7b2000000560 by thread T5: #0 Counter.next() <null>:2 (a.out:x86_64+0x1000047f8) #1 (1) suspend resume partial function for worker(identity:counters:numIterations:) <null>:2 (a.out:x86_64+0x100005961) #2 swift::runJobInEstablishedExecutorContext(swift::Job*) <null>:2 (libswift_Concurrency.dylib:x86_64+0x280ef) Previous write of size 8 at 0x7b2000000560 by main thread: #0 Counter.init(maxCount:) <null>:2 (a.out:x86_64+0x1000046af) #1 Counter.__allocating_init(maxCount:) <null>:2 (a.out:x86_64+0x100004619) #2 runTest(numCounters:numWorkers:numIterations:) <null>:2 (a.out:x86_64+0x100006d2e) #3 swift::runJobInEstablishedExecutorContext(swift::Job*) <null>:2 (libswift_Concurrency.dylib:x86_64+0x280ef) #4 main <null>:2 (a.out:x86_64+0x10000a175) ``` New edge with this change: ``` [4357150208] allocate task 0x7b3800000000, parent = 0x0 [4357150208] creating task 0x7b3800000000 with parent 0x0 [4357150208] tsan_release on 0x7b3800000000 <<< new release edge [139088221442048] tsan_acquire on 0x7b3800000000 [139088221442048] trying to switch from executor 0x0 to 0x7ff85e2d9a00 [139088221442048] switch failed, task 0x7b3800000000 enqueued on executor 0x7ff85e2d9a00 [139088221442048] enqueue job 0x7b3800000000 on executor 0x7ff85e2d9a00 [139088221442048] tsan_release on 0x7b3800000000 [139088221442048] tsan_release on 0x7b3800000000 [4357150208] tsan_acquire on 0x7b3800000000 counters: 1, workers: 1, iterations: 1 [4357150208] allocate task 0x7b3c00000000, parent = 0x0 [4357150208] creating task 0x7b3c00000000 with parent 0x0 [4357150208] tsan_release on 0x7b3c00000000 <<< new release edge [139088221442048] tsan_acquire on 0x7b3c00000000 [4357150208] task 0x7b3800000000 waiting on task 0x7b3c00000000, going to sleep [4357150208] tsan_release on 0x7b3800000000 [4357150208] tsan_release on 0x7b3800000000 [139088221442048] getting current executor 0x0 [139088221442048] tsan_release on 0x7b3c00000000 ... ``` rdar://78932849 * Add static_cast<Job *>() * Move TSan release edge to swift_task_enqueueGlobal() Move the TSan release edge from `swift_task_create_commonImpl()` to `swift_task_enqueueGlobalImpl()`. Task creation itself is not an event that needs synchronization, but rather that task creation "happens before" execution of that task on another thread. This edge is usually added when the task is scheduled via `swift_task_enqueue()` (which then usually calls `swift_task_enqueueGlobal()`). However, not all task scheduling goes through the `swift_task_enqueue()` funnel as some places call the more specific `swift_task_enqueueGlobal()` directly. So let's annotate this function (duplicate edges aren't harmful) to ensure we cover all schedule events, including newly-created tasks (our original problem here). rdar://78932849 Co-authored-by: Julian Lettner <[email protected]>

@thin

Case #1. Literal zero = natural alignment %1 = integer_literal $Builtin.Int64, 0 %2 = builtin "uncheckedAssertAlignment" (%0 : $Builtin.RawPointer, %1 : $Builtin.Int64) : $Builtin.RawPointer %3 = pointer_to_address %2 : $Builtin.RawPointer to [align=1] $*Int Erases the `pointer_to_address` `[align=]` attribute: Case #2. Literal nonzero = forced alignment. %1 = integer_literal $Builtin.Int64, 16 %2 = builtin "uncheckedAssertAlignment" (%0 : $Builtin.RawPointer, %1 : $Builtin.Int64) : $Builtin.RawPointer %3 = pointer_to_address %2 : $Builtin.RawPointer to [align=1] $*Int Promotes the `pointer_to_address` `[align=]` attribute to a higher value. Case #3. Folded dynamic alignment %1 = builtin "alignof"<T>(%0 : $@thin T.Type) : $Builtin.Word %2 = builtin "uncheckedAssertAlignment" (%0 : $Builtin.RawPointer, %1 : $Builtin.Int64) : $Builtin.RawPointer %3 = pointer_to_address %2 : $Builtin.RawPointer to [align=1] $*T Erases the `pointer_to_address` `[align=]` attribute.

Update DifferentiableProgrammingImplementation.md

While trying to reuse the liveness-points analysis originally in DI for injecting actor hops for more general purposes, Pavel and I discovered that the point at which we are injecting the hops might not have fully-computed the liveness information. That appears to be the case because we were computing the fully-initialized points before having processed destroy/releases of TheMemory. While this most likely had no influence on the actor hop injection, it does affect what the outgoing AvailabilitySet contains for a block. In particular, for this example: ```swift struct X { init(cond: Bool) { var _storage: (name: String, age: Int) _storage.name = "" if cond { _storage.age = 30 } else { _storage.age = 40 } } } ``` But because we are determine the full initialization points before processing the destroy, the liveness analysis doesn't iterate to correctly determine the out-availability of block 1 and 3 (corresponding to the then and else blocks in the example above). Here's the debug output showing that issue: ``` *** Definite Init looking at: %5 = mark_uninitialized [var] %4 : $*(name: String, age: Int) // users: %37, %12, %22, %32 Get liveness 0, #1 at assign %11 to %13 : $*String // id: %14 Get liveness 1, #1 at assign %21 to %23 : $*Int // id: %24 Get liveness for block 1 Iteration 0 Result: (yn) Get liveness 1, #1 at assign %31 to %33 : $*Int // id: %34 Get liveness for block 3 add block 2 to worklist Iteration 0 Block 2 out: (yn) Iteration 1 Block 2 out: (yn) Result: (yn) full-init-finder: rejecting bb0 b/c non-Yes OUT avail full-init-finder: rejecting bb1 b/c non-Yes OUT avail full-init-finder: rejecting bb2 b/c no non-load uses. full-init-finder: rejecting bb3 b/c non-Yes OUT avail full-init-finder: rejecting bb4 b/c no non-load uses. Get liveness 0, #2 at destroy_addr %5 : $*(name: String, age: Int) // id: %37 Get liveness for block 4 add block 3 to worklist add block 1 to worklist Iteration 0 Block 1 out: (yy) Block 3 out: (yy) Iteration 1 Block 1 out: (yy) Block 3 out: (yy) Result: (yy) ``` So, this patch basically just sinks the computation so it happens after, so that we force the incremental liveness analysis to also consider the liveness at the point of the destroy, but before having done any other transformations or modifications to the CFG to handle a destroy of something partially initialized.

CI

zhuowei force-pushed the swiftwasm branch from d6bd630 to 399b5ae Compare April 29, 2019 21:20

zhuowei closed this Apr 29, 2019

pull bot pushed a commit that referenced this pull request Apr 2, 2021

Merge pull request swiftlang#36714 from rintaro/ide-completion-batcht…

876a0ee

…est1 [CodeCompletion] Migrate some tests to batch completion test #1

pull bot pushed a commit that referenced this pull request Dec 9, 2021

Merge pull request #1 from philipturner/philipturner-patch-4-pt-1

b5e151c

Update DifferentiableProgrammingImplementation.md

kateinoigakukun added a commit that referenced this pull request Jun 28, 2023

Merge pull request #1 from swiftwasm/katei/ci

bede4d5

CI

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Add wasm branch scheme to update-checkout-config #1

Add wasm branch scheme to update-checkout-config #1

Uh oh!

MaxDesiatov commented Apr 29, 2019

Uh oh!

zhuowei commented Apr 29, 2019

Uh oh!

zhuowei commented Apr 29, 2019

Uh oh!

MaxDesiatov commented Apr 29, 2019 •

edited

Loading

Uh oh!

Uh oh!

Add wasm branch scheme to update-checkout-config #1

Add wasm branch scheme to update-checkout-config #1

Uh oh!

Conversation

MaxDesiatov commented Apr 29, 2019

Uh oh!

zhuowei commented Apr 29, 2019

Uh oh!

zhuowei commented Apr 29, 2019

Uh oh!

MaxDesiatov commented Apr 29, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

MaxDesiatov commented Apr 29, 2019 •

edited

Loading