[SR-1264] Swift doesn't build on Android

[swift-ci]

Previous ID	SR-1264
Radar	None
Original Reporter	FlyingSnake (JIRA User)
Type	Bug
Status	Closed
Resolution	Done

Attachment: Download

Environment

Ubuntu 14.04 (on Vagrant, VirtualBox, Mac OS X)

Additional Detail from JIRA

Votes	0
Component/s	Compiler
Labels	Bug, Android, Linux
Assignee	@modocache
Priority	Medium

md5: c10475de3cf77c747adcf7824abe13c5

Issue Description:

strong textHi,

I've setup a vagrant box with Ubuntu 14.04 and followed the instructions at https://github.com/apple/swift/blob/master/docs/Android.md. The compilation fails at the linker stage showing the error

/usr/bin/ld: unrecognised emulation mode: armelf_linux_eabi
Supported emulations: elf_x86_64 elf32_x86_64 elf_i386 i386linux elf_l1om elf_k1om i386pep i386pe

The buildbot_linux_arm7 command works fine without any issues.

I've attached the build log and I would really appreciate any help in getting this fixed.

[belkadan]

cc @modocache

[modocache]

When using the latest Android NDK release, android-ndk-r11c, I encounter the exact same error message during compilation. Full log: https://gist.github.com/modocache/4bfce5c26319b7e0cc11e643a126bab5

I figure that either the problem is due to the NDK release version (I've only tested this on android-ndk-r10e, the latest release at the time I began working on Android), or that I had some sort of local modification to my copy of android-ndk-r10e that made it work. I'm currently investigating.

FlyingSnake (JIRA User), could you please tell me what version of the NDK you're using? You may be able to tell by looking at the CHANGELOG.md and source.properties files at the top level of your NDK directory. On older versions, you can check the RELEASE.txt file.

Until I can fix this issue, I'd recommend using android-ndk-r10e.

[modocache]

FlyingSnake (JIRA User): In the immediate short-term, you should be able to get the Android build working with android-ndk-r11c (the latest NDK version) with two small modifications:

1. Supply --android-ndk-toolchain-version 4.9 to the build script. This defaults to 4.8, which appears to be gone in the latest NDK. Here's the new invocation you should use, in full:

$ ./swift/utils/build-script \
    -R \
    --android \
    --android-ndk /vagrant/android-sdk-linux/ndk-bundle \
    --android-ndk-version 21 \
    --android-icu-uc /vagrant/libiconv-libicu-android/armeabi-v7a/libicuuc.so \
    --android-icu-uc-include /vagrant/libiconv-libicu-android/armeabi-v7a/icu/source/common \
    --android-icu-i18n /vagrant/libiconv-libicu-android/armeabi-v7a/libicui18n.so \
    --android-icu-i18n-include /vagrant/libiconv-libicu-android/armeabi-v7a/icu/source/i18n/ \
    --android-ndk-toolchain-version 4.9

2. The latest NDK now defines malloc_usable_size. I'll submit a pull request soon to get this working. For now, you can make the following source modification to your Swift checkout (easier to read version here):

diff --git a/stdlib/public/stubs/LibcShims.cpp b/stdlib/public/stubs/LibcShims.cpp
index cb750f1..a3164fa 100644
--- a/stdlib/public/stubs/LibcShims.cpp
+++ b/stdlib/public/stubs/LibcShims.cpp
@@ -52,7 +52,7 @@ int _swift_stdlib_close(int fd) { return close(fd); }
 #if defined(__APPLE__)
 #include <malloc/malloc.h>
 size_t _swift_stdlib_malloc_size(const void *ptr) { return malloc_size(ptr); }
-#elif defined(__GNU_LIBRARY__) || defined(__CYGWIN__)
+#elif defined(__GNU_LIBRARY__) || defined(__CYGWIN__) || defined(__ANDROID__)
 #include <malloc.h>
 size_t _swift_stdlib_malloc_size(const void *ptr) {
   return malloc_usable_size(const_cast<void *>(ptr));
@@ -62,13 +62,6 @@ size_t _swift_stdlib_malloc_size(const void *ptr) {
 size_t _swift_stdlib_malloc_size(const void *ptr) {
   return malloc_usable_size(const_cast<void *>(ptr));
 }
-#elif defined(__ANDROID__)
-extern "C" {
-extern size_t dlmalloc_usable_size(void*);
-}
-size_t _swift_stdlib_malloc_size(const void *ptr) {
-  return dlmalloc_usable_size(const_cast<void *>(ptr));
-}
 #else
 #error No malloc_size analog known for this platform/libc.
 #endif

[modocache]

I'd appreciate some help from any Android developers out there that make frequent use of the NDK. Specifically:

• Do you use the latest version of the NDK?

• Is there ever a reason not to use the latest version of the Android NDK?

• Which version would you expect Swift to support?

• Should Swift attempt to support multiple versions? This seems like it would take quite a bit of effort, so I'd like to avoid this if possible.

[swift-ci]

Comment by Sam Khawase (JIRA)

Hi Brian,

Thanks a lot for your help.
You've guessed correctly, I'm using android-ndk-r11c.

source.properties
Pkg.Desc = Android NDK
Pkg.Revision = 11.2.2725575

CHANGELOG.md
Hotfix r11c

I've applied your fix and started a clean build. I'll report back once it completes.

[swift-ci]

Comment by Sam Khawase (JIRA)

Hi Brian,

I use NDK regularly. The project is a fairly complex cross-platform app(s) with UI layers in native (Android/iOS) and almost 80-85% code base is C++ (I dream of replacing C++ with Swift one day)

Do you use the latest version of the NDK?
No. we use r11c

Is there ever a reason not to use the latest version of the Android NDK?
We use the NDK r11c because the latest NDK v12 broke our build and we had to revert back to r11c.

Which version would you expect Swift to support?
I would expect support for latest 2 NDK versions, only if it's easy to maintain.

Should Swift attempt to support multiple versions? This seems like it would take quite a bit of effort, so I'd like to avoid this if possible.
I think Swift should support a version that's not far from the latest, and just stick to it. As Android build works only in Linux, it's fairly easy to spin up a Vagrant/Docker/LXD box to build Swift toolchain. It would be great if we could provide a disclaimer in the Android.md manual for Swift. That way people won't try to compile against incompatible versions. (like I did).

[modocache]

Excellent, thanks FlyingSnake (JIRA User). Here's a pull request that should resolve this issue: #2244 I think this and #2242 would unbreak the Swift Android build on master.

My apologies: ideally the build wouldn't have been broken for a day in the first place. I'm working on supporting the test suite on Android in #1714 and after that's done I'm hoping to set up some sort of local CI so that I can report failures faster in the future.

[swift-ci]

Comment by Sam Khawase (JIRA)

Hi @modocache

I've tried the build and unfortunately it's still broken. I think the culprit is the libiconv-libicu-android which doesn't get compiled under NDK r11c. I've attached the error logs and will try to compile using NDK r10e and see if I can build Swift. Are there prebuilt binaries for Swift for Android, which can be made available to everyone?

Also can we spin up an instance of Travis CI for Android builds, since they are free for open-source projects?

libicu-build.log swift-build-r11c.log

[modocache]

Thanks for the additional attachments! Yes, the libicu build script needs to be updated (see https://twitter.com/neonacho/status/722750280161476609). I believe it'll work if you remove the check described in that tweet--will push a fix to that repo shortly.

[modocache]

FlyingSnake (JIRA User), I just merged SwiftAndroid/libiconv-libicu-android#2 into SwiftAndroid/libiconv-libicu-android. That build now works with android-ndk-r11c.

Travis CI is a good idea. We unfortunately can't use it on the main apple/swift repository, though (I do want to think whether we can get Swift on Android incorporated into the main CI some day). We could use Travis if we set up a mirror repository that periodically rebased on top of master. Setting that up would take a bit of effort... any help there would be much appreciated.

And yes, packaging binaries is a great idea. I created a GitHub release, you can download the build products from there: https://github.com/SwiftAndroid/libiconv-libicu-android/releases/tag/android-ndk-r11c

[swift-ci]

Comment by Sam Khawase (JIRA)

Man I am seriously impressed by your dedication to this project. Not only you are a kick-ass programmer, but you've got crazy enthusiasm in addition!

I'll get back to compiling Swift4Android and hopefully will succeed this time :-D. I'll probably create a vagrant box with Ansible provisioning to help people spin up a Swift-build-bot OR investigate how to create the builtbot preset like buildbot_linux_armv7

Thanks a lot for all your help!

[modocache]

Haha, thanks!! 🙂

Also, thank you for kicking the tires on this the Swift Android build. Only a handful of people have attempted to build it since it was merged. Feedback at this early stage is an important part of making it stable for everyone.

[swift-ci]

Comment by Sam Khawase (JIRA)

I'll try to get the Travis CI build up once my vagrant swift build-bot is ready. Once we've a CI for Swift/Android, people will know we mean serious business 😃

[modocache]

FYI: If you pull from apple/swift master, you'll encounter an entirely different build error. Apply #2252 in order to fix it. Our CI strategy can't come soon enough! 🙂

[swift-ci]

Comment by Sam Khawase (JIRA)

Sigh, one more build failure on NDK r11c. I did a clean build of libicu and latest Swift from upstream, and it still failed at linker stage. I tried re-running the build script and got more details about the error.

/vagrant/android-sdk-linux/ndk-bundle/toolchains/arm-linux-androideabi-4.9/prebuilt/linux-x86_64/arm-linux-androideabi/bin/ld: error: cannot find -licuuc

/vagrant/android-sdk-linux/ndk-bundle/toolchains/arm-linux-androideabi-4.9/prebuilt/linux-x86_64/arm-linux-androideabi/bin/ld: error: cannot find -licui18n

Looks like the linker isn't happy with the libicu that we're passing as a parameter.

swift-build-1461231397.log and swift-build-1461233286.log

I noticed that you're using Ubuntu 15.10 and I'll try to spin up another vagrant box to verify if it builds on 15.10 (I'm on Trusty LTS).

Once I get a solid build, I'll start a Travis CI job and hopefully we'll have better integration.