How to use host volume in OSX by docker-machine (boot2docker)

[dsxack]

I use boot2docker through virtualbox via docker-machine. I seen #44, but doesn't find answer, how resolve this

docker-compose.yml:

mysql:
  image: mysql
  volumes:
    - ./volumes/mysql:/var/lib/mysql
  environment:
    - MYSQL_ALLOW_EMPTY_PASSWORD=yes

output:

mysql_1    | Running mysql_install_db
mysql_1    | 2015-09-10 21:28:40 0 [Note] /usr/sbin/mysqld (mysqld 5.6.26) starting as process 17 ...
mysql_1    | 2015-09-10 21:28:40 17 [Note] InnoDB: Using atomics to ref count buffer pool pages
mysql_1    | 2015-09-10 21:28:40 17 [Note] InnoDB: The InnoDB memory heap is disabled
mysql_1    | 2015-09-10 21:28:40 17 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
mysql_1    | 2015-09-10 21:28:40 17 [Note] InnoDB: Memory barrier is not used
mysql_1    | 2015-09-10 21:28:40 17 [Note] InnoDB: Compressed tables use zlib 1.2.7
mysql_1    | 2015-09-10 21:28:40 17 [Note] InnoDB: Using Linux native AIO
mysql_1    | 2015-09-10 21:28:40 17 [Note] InnoDB: Not using CPU crc32 instructions
mysql_1    | 2015-09-10 21:28:40 17 [Note] InnoDB: Initializing buffer pool, size = 128.0M
mysql_1    | 2015-09-10 21:28:40 17 [Note] InnoDB: Completed initialization of buffer pool
mysql_1    | 2015-09-10 21:28:40 7f2930843720  InnoDB: Operating system error number 13 in a file operation.
mysql_1    | InnoDB: The error means mysqld does not have the access rights to
mysql_1    | InnoDB: the directory.
mysql_1    | 2015-09-10 21:28:40 7f2930843720  InnoDB: Operating system error number 13 in a file operation.
mysql_1    | InnoDB: The error means mysqld does not have the access rights to
mysql_1    | InnoDB: the directory.
mysql_1    | 2015-09-10 21:28:40 17 [ERROR] InnoDB: Creating or opening ./ibdata1 failed!
mysql_1    | 2015-09-10 21:28:40 17 [ERROR] InnoDB: Could not open or create the system tablespace. If you tried to add new data files to the system tablespace, and it failed here, you should now edit innodb_data_file_path in my.cnf back to what it was, and remove the new ibdata files InnoDB created in this failed attempt. InnoDB only wrote those files full of zeros, but did not yet use them in any way. But be careful: do not remove old data files which contain your precious data!
mysql_1    | 2015-09-10 21:28:40 17 [ERROR] Plugin 'InnoDB' init function returned error.
mysql_1    | 2015-09-10 21:28:40 17 [ERROR] Plugin 'InnoDB' registration as a STORAGE ENGINE failed.
mysql_1    | 2015-09-10 21:28:40 17 [ERROR] Unknown/unsupported storage engine: InnoDB
mysql_1    | 2015-09-10 21:28:40 17 [ERROR] Aborting
mysql_1    | 
mysql_1    | 2015-09-10 21:28:40 17 [Note] Binlog end
mysql_1    | 2015-09-10 21:28:40 17 [Note] /usr/sbin/mysqld: Shutdown complete
mysql_1    | 
mysql_1 exited with code 141

[bryanrossUK]

this comment on #44 does a good job of explaining the problem. Unfortunately, there doesn't seem to be a good solution other than storing your database files in a Docker Data Volume Container (rather than a directory on your computer) or change your Virtualbox/boot2docker configuration to have more permissive file access.

[4thAce]

I tried giong the data volume container route but it hit the same InnoDB error. It looks like for dev environments we will just tell our users to provision mysql using homebrew instead of bringing up a container.

[motin]

The solution is to make sure that mysql runs using the same user and group ids as your local OSX user.

Create the following script:

#!/bin/bash
set -e # fail on any error

echo '* Working around permission errors locally by making sure that "mysql" uses the same uid and gid as the host volume'
TARGET_UID=$(stat -c "%u" /var/lib/mysql)
echo '-- Setting mysql user to use uid '$TARGET_UID
usermod -o -u $TARGET_UID mysql || true
TARGET_GID=$(stat -c "%g" /var/lib/mysql)
echo '-- Setting mysql group to use gid '$TARGET_GID
groupmod -o -g $TARGET_GID mysql || true
echo
echo '* Starting MySQL'
chown -R mysql:root /var/run/mysqld/
/entrypoint.sh mysqld --user=mysql --console

Save it as localdb-run.sh or similar, mount it into the mysql container and use it as your starting command.

[dsxack]

@motin, thanks, i will try this solution at weekends.

[motin]

@dsxack Here is a sample implementation: https://github.com/neam/docker-stack/tree/develop/stacks/neam/debian-php-nginx.dna-project-base/stack

[dbaba]

I'd had the same error.
@motin The script worked for me. Thanks!

In docker-compose.yml,

(snip)
  entrypoint: /localdb-run.sh
  volumes:
    - ./path/to/localdb-run.sh:/localdb-run.sh
(snip)

This should work with docker-compose.

[motin]

@dbaba Great that it worked for you as well :) Here is a full docker-compose implementation with the script:

docker-compose.yml
stack/localdb/run.sh

Full explanation:

Step 1: Add the script run.sh somewhere in your project:

#!/bin/bash
set -e

# Script to workaround docker-machine/boot2docker OSX host volume issues: https://github.com/docker-library/mysql/issues/99

echo '* Working around permission errors locally by making sure that "mysql" uses the same uid and gid as the host volume'
TARGET_UID=$(stat -c "%u" /var/lib/mysql)
echo '-- Setting mysql user to use uid '$TARGET_UID
usermod -o -u $TARGET_UID mysql || true
TARGET_GID=$(stat -c "%g" /var/lib/mysql)
echo '-- Setting mysql group to use gid '$TARGET_GID
groupmod -o -g $TARGET_GID mysql || true
echo
echo '* Starting MySQL'
chown -R mysql:root /var/run/mysqld/
/entrypoint.sh mysqld --user=mysql --console

Step 2: Change the command in docker-compose.yml to run this script instead of the ordinary command.

Example:

# Local database server to mimic a cloud database
localdb:
  image: mysql:5.6.27
  volumes:
    - ./stack/localdb/.db/mysql:/var/lib/mysql:rw
    - ./stack/localdb/:/stack/localdb:rw
  ports:
    - "3306"
  environment:
    MYSQL_ROOT_PASSWORD: "local-mysql-pass"
  command: "/stack/localdb/run.sh"

[viktorsteinwand]

@motin @dbaba: Many thanks! It works.

[yangjm]

@motin Thank you your script and yml works perfectly.

[IanEdington]

Thanks @motin

I know this was probably clear to everyone already but when I tried to create a container with this script I had a number of issues with permissions and adding the file. This was my solution (incase someone like me has this problem in the future).

Project Folder
 ├─ localdb-run.sh  - from @motin
 └─ Dockerfile

Dockerfile

FROM mysql
MAINTAINER Ian Edington "IanEdington@gmail.com"

ENV MYSQL_ROOT_PASSWORD password
ENV MYSQL_PASSWORD password
ENV MYSQL_DATABASE databasename

# this need to stay the same for script to work
ENV MYSQL_USER mysql

COPY ./localdb-run.sh /
RUN chmod 755 /localdb-run.sh

ENTRYPOINT ["/localdb-run.sh"] 

Build and run container

docker build -t IanEdington/mysql .
docker run --name mysql -v /Users/path/to/mysql/directory:/var/lib/mysql -d IanEdington/mysql .

[roocell]

Thanks @motin and @IanEdington - works great.
I've been spending 10X more time fighting docker<->osx issues than I've been actually doing what I've intended to do.

[zmoon111]

thx, it does work!

[sebastianvilla]

Quick note: you need to chmod +x localdb-run.sh on the host (your computer) or use "sh path/to/localdb-run.sh" in your command or entrypoint

[jstdoit]

It works for me. I'm using a MacBook pro

[gharcombe-minson]

Thank you @motin - amazing fix and unblocks my efforts to improve our dev environment!