19.12.2014

Türchen 19: Kickstart your Magento Dev System with Vagrant

Vagrant is a tool that allows portable virtual development environments, using virtual machines. If multiple team members work on different devices on a project, a unified environment can be ensured. It only has to be set up once and can be reproduced as often as needed.

But it is also interesting to single developers who work on multiple projects. Often the different production systems have different PHP versions installed or they need specific extensions and system configurations. Especially with PHP where much behaviour depends on global configuration, you often get errors due to different systems.

With Vagrant you can define the whole environment for each project and also put this definition under version control. Ideally this resembles the production system as close as possible. Also, your own system is less cluttered up with software that you needed for some past project.

Although Vagrant manages virtual machines, it is not a VM provider itself, it works with different providers, such as VirtualBox, VMware and even AWS.

Vagrant also works with many different provisioning tools, i.e. tools for automated system setup, like Puppet, but also comes with very simple built-in provisioners to copy files and run shell scripts. You can combine all of those.

To summarize the advantages of Vagrant:

Operating system and global configuration as close as possible to the production server
Reproducible environment
Versioned system configuration
Automated setting up of new environments

But isn't this, like, super-complicated?

You might be reluctant to learning new technologies (Vagrant, Puppet) and fear the amount of initial work to set everything up. Then I have good news: I already did most of it for you. In this article I introduce the /> integer_net Magento Vagrantbox that should get you started in no time, for new projects as well as for existing projects.

Some technical details beforehand

Terminology:

Box, Vagrantbox: The virtual machine, managed by Vagrant
Host: The computer that runs Vagrant
Provisioning: The automatic setup that gets executed on the first start and when requested explicitly, after configuration changes.

File synchronization

Somehow the project files from the host need to be available on the box. Vagrant offers different synchronization strategies, currently NFS, Rsync, SMB and VirtualBox Shared Folders. All of them have their pros and cons but because file system performance is essential for Magento development, the best choice is Rsync: files from the host are synchronized to the box in real time by a file system watcher process. This way you work on the native file system of the host but the web server uses the native file system of the VM.

Puppet and PuPHPet

For provisioning we use a combination of Puppet and shell scripts, built with the GUI configurator PuPHPet. This makes it easy for you to change server configuration without knowing the Puppet language, because it is all in one human readable YAML file

puphpet/config.yaml

which you can either edit manually or drag and drop into the configurator at http://puphpet.com to edit it. You will get a big zip file to download but all you need from it is the config.yaml to replace your original version. The PuPHPet configurator is pretty self-explanatory. If you want to know more, it has a good "About everything" page with some background, also about Vagrant.

VM Performance

Pay attention to the memory (RAM) you assign to your boxes, especially when you start multiple boxes at the same time, because it is subtracted from the host. I'd recommend to start with 512M for Magento and increase only when necessary. We use VirtualBox basically because it's the standard provider and it's entirely free, but honestly it's also the least performant one. Consider VMware if you need a performance boost. With PuPHPet you can change the provider with a single click.

Magento setup

Magento will be set up with shell scripts, that can be found in

puphpet/files/exec-once

and will run in alphabetical order on the first provisioning. When you add new scripts to the directory or change existing scripts, those will run on the next provisioning.

Let's set it up!

1. Install required Software

Vagrant: Download from https://www.vagrantup.com/downloads.html
Virtualbox: Get the VirtualBox platform packages for your OS and the VirtualBox Extension Pack, both from https://www.virtualbox.org/wiki/Downloads
Puppet: depends on your OS
- Windows: Download latest version from https://downloads.puppetlabs.com/windows/?C=M;O=D
- Mac: Download latest version from https://downloads.puppetlabs.com/mac/?C=M;O=D
- Linux: Use your favourite package manager. See https://docs.puppetlabs.com/guides/puppetlabs_package_repositories.html for details
(only on Windows) Cygwin: For our setup we need a Unix shell with rsync and SSH support. Get Cygwin from https://cygwin.com/install.html and select the packages “rsync” and “openssh” with “” during installation.

2. Download our project boilerplate

Download and extract https://github.com/schmengler/magento-boilerplate/archive/master.zip – this will be your project directory. Navigate to the directory and run

./up –

You will be asked for a domain and IP address. Try „webguys2014.local“ and 192.168.56.111. Add this combination to your systems

/etc/hosts

file. You can also choose between different project structure templates. Currently these are:

Modman+Composer: choose this if you manage your source with modman to separate it from core code. Using Composer for external dependencies is also supported but optional.
All-in-one: choose this if all your source code lives under the Magento web root. You will be asked for a directory for your sources, which must be a subdirectory of the project directory. Choose „src“ if you are not sure.

Now you have the option to continue immediately, this will set up a new Magento installation with some default settings. But let's wait with that for a minute to see what needs to be adjusted most of the time.

EITHER 3a. Choose parameters for new installation

If you used the Composer+Modman template, Magento gets installed with Composer by default. You can change the version in composer.json. You can also add or remove extensions there to your liking. If you want to install the sample data as well, look into

puphpet/files/exec-once/90-magento.sh

and adjust the n98-magerun install command (change

--installSampleData=no

--installSampleData=yes

). If you used the All-in-one template, you can change the Magento version and choose to install the sample data in

<span style="font-weight: normal">puphpet/files/exec-once/70-install-magento.sh</span>

OR 3b. Add existing source code

If you used the Composer+Modman template, first replace the composer.json file with your own or remove it if you don't use Composer. If you use Composer, make sure, you require „aoepeople/composer-installers“ - the box is configured to work with „Fabrizio's Lightweight Approach“ (see http://fbrnc.net/blog/2014/11/how-to-install-a-magento-module). The following composer configuration is required for the composer installers to work:

"extra":{<br>
       "installer-paths": {<br>            "www/"
: [ "type:magento-source" ],<br>           
".modman/{$name}/" : [ "type:magento-module" ]<br>
       }<br>    },<br>

And this one is recommended to automatically trigger modman deployment:

"scripts":{<br>
       "post-install-cmd": "cd /home/vagrant &&
modman deploy-all --force",<br>        "post-update-cmd":
"cd /home/vagrant && modman deploy-all --force"<br>
   }

If you reference private repositories in composer.json you may copy your private SSH key to puphpet/files/dot/.ssh – the file name must end in _id_rsa, then it will be automatically registered with the SSH agent on the box. Then add your own modules under

src

and import them in

src/modman

like this:

@import MyModule

Note that absolutely no custom code will be in the www directory, only the untouched Magento core (the directory is not synchronized to the box, just sits there for browsing and debugging). If parts of your code are not deployed with modman, you should still make it a module under

src

, for example as

src/Base

with a modman file like this:

@shell rsync -avz --update --omit-dir-times
--no-o --no-p --no-g --exclude="modman" $MODULE/. $PROJECT

If you used the All-in-one template, add the code to the directory you specified before. If you want or have to use a separate repository, you can do this with a Git submodule as well. To import an existing database, place a compressed dump (

*.sql.gz

) into

systemstorage/live/database/

- specific Magento settings for the development system may be configured in

puphpet/files/exec-once/91-magento-config.sh

– the default base URL and a few other default settings are already there, extend it as needed.

4. Provisioning...

Now that everything is prepared, you can run the „up“ script again (or press „y“ if it's still waiting for your input) … and wait … The first start is going to take quite a while, so go grab a coffee or read the rest of the article while waiting. Don't worry about single lines of red messages scrolling by, they are usually no problem. Only if you see whole pages of red, a closer look might be required. When everything was successful, you will be greeted with this menu:

You will see the rsync watcher working and have some shortcuts available:

L: Login to the box with SSH (same as „
```
vagrant
    ssh
```
“)
M: Login to the box with a direct call of „
```
n98-magerun
    shell
```
“. Useful if you only need magerun commands (like „cache:flush“). Exit the console with Ctrl-D.
H: Shut down the box (same as „
```
vagrant
    halt
```
“, but also kills the rsync watcher first)
S: Suspend the box (same as „
```
vagrant
    suspend
```
“, again with killing the rsync watcher first)
R: Restart the rsync watcher process (it sometimes crashes, for example after the host has been in power-saving mode)
P: Provision the box while it runs, for example after a change in
```
config.yaml
```
or if there are new/changed files in
```
puphpet/files/exec-once
```
.

But nothing of should be necessary right now, just leave the window open and...

5. Fire up your browser

Browse to http://webguys2014.local/. If you are greeted with the Magento installation wizard, Magentos

app/etc/local.xml

file is incomplete. Now you have two options:

Follow the installation wizard (DB access is root/root)
Adjust
```
etc/local.xml
```
– this file is linked directly into the Magento installation, so that you can put it into version control. But the existing file is incomplete, you need to add install date and encryption key. It makes sense to copy these from an existing system. Then you need to flush the Magento cache (
```
n98-magerun cache:flush
```
in the box) and open the store again.

VoilÃ :

Access the backend at http://webguys2014.local/admin with username "admin" and password "test123" At http://webguys2014.local:1080/ you will find the Mailcatcher web interface. All outgoing mails are captured there (instead of really sending them)

Configuration / usage

The „up“ script

After the initial configuration, you can still use "up" as a convenience script to start the box and access some common commands with a single keystroke from the startup menu. But if you prefer, you can use the Vagrant commands directly as well:

```
vagrant up
```
to start a box
```
vagrant rsync-auto
```
to start the rsync watcher
```
vagrant ssh
```
to log in via SSH
```
vagrant provision
```
to provision a running box
```
vagrant halt
```
to shutdown a running box

SSH connection from other tools (IDE, MySQL client, ...)

You can run

vagrant ssh-config

to see get information on how to access the box. It will show you something like:

Host default
  HostName 127.0.0.1
  User vagrant
  Port 2222
  UserKnownHostsFile /dev/null
  StrictHostKeyChecking no
  PasswordAuthentication no
  IdentityFile C:/Users/fs/Projekte/intern/webguys2014/puphpet/files/dot/ssh/id_rsa
  IdentityFile C:/Users/fs/Projekte/intern/webguys2014/puphpet/files/dot/ssh/insecure_private_key
  IdentitiesOnly yes
  LogLevel FATAL
  ForwardAgent yes

Vagrant uses port forwarding from

127.0.0.1:2222

to port

of the box. The thing is, this port will change, if it is already in use, like from another box. So I prefer using the IP of the box (e.g.

192.168.56.111

) with port

The username is always "vagrant" and the private key

puphpet/files/dot/ssh/insecure_private_key

Crontab

The file

etc/crontab

is installed as crontab on provisioning. This way the cron configuration can be versioned as well. The placeholder

{DOMAIN}

will be replaced with the content of

etc/domain

which was generated on the first setup we did before.

Magento configuration

Configuration of Magento that is specific to the development system should be done in

puphpet/files/exec-once/91-magento-config.sh

instead of manually in the backend so that it will be done automatically the next time somebody installs the box. Just add the necessary n98-magerun commands, for example to add a different base URL for a second store view:

n98-magerun config:set --scope="stores" --scope-id=2 web/unsecure/base_url http://en.webguys2014.local/

Usage of the systemstorage directory

With

/vagrant/update_systemstorage.sh

(executed in the box) a snapshot of database and media directory gets saved in

systemstorage/devbox

. This way you also can put it into version control or share it with other developers. However, due to the big amount of binary data I would not recommend to use the project Git repository. Thanks to Fabrizio Branca for the original version in https://github.com/AOEpeople/Magento_SampleProject!

Dotfiles

User specific Unix configuration files, the so called dotfiles, are copied on provisioning from

puphpet/files/dot

to the home directory in the box. You can replace the existing default files with your own. Many developers maintain their personal dotfiles, so it might be sensible to exclude the directory from version control (i.e. add it to .gitignore).

Xdebug

Xdebug is configured with the default IDE key "xdebug". You can change this together with other Xdebug configuration in

puphpet/config.yaml

. To debug CLI scripts, you can use

xdebug

as an alias for

php

. In this case, the IDE key "xdebug" is hard coded. Examples:

xdebug -d memory_limit=1024M www/shell/indexer.php

xdebug vendor/phpunit/phpunit/composer/bin/phpunit --filter IntegerNet_Solr

Don't forget to setup path mapping in your IDE to use the debugger.