[Nix-dev] Tips on deploying a Scala Play application
4levels
4levels at gmail.com
Wed Jul 6 21:03:54 CEST 2016
Hi Teo,
My previous email is too long and being considered for moderation by the
nix-dev mailing list.
I forgot to also include the files based on your code. This is where the
build fails:
./mancloud-play-wrapper.nix:37:17 - the script line of the systemd.service
declaration
*mancloud-play-wrapper.nix* is the wrapper script from your deployment
subfolder
{ stdenv, writeText, writeScript, makeWrapper
, bash
, mancloud-play
, port
, lib
, useRecaptcha ? false
, recaptchaSiteKey ? null
, recaptchaSecretKey ? null
, reverseProxyIp ? null
}:
assert useRecaptcha -> (recaptchaSiteKey != null && recaptchaSecretKey != null);
let
configFile = writeText "mancloud-play.conf"
''
include "application.conf"
${lib.optionalString (reverseProxyIp != null)
"play.http.forwarded.trustedProxies += ['${reverseProxyIp}']"}
${lib.optionalString useRecaptcha
''
captcha {
provider = "recaptcha"
recaptcha {
siteKey = ${recaptchaSiteKey}
secretKey = ${recaptchaSecretKey}
}
}
''
}
'';
in {
systemd.services = {
mancloud-play = {
wantedBy = [ "multi-user.target" ];
requires = [ ];
after = [ "network.target" ];
script = "exec ${mancloud-play}/bin/mancloud-play
-Dconfig.file=${configFile} -Dhttp.port=${toString port}";
};
};
}
*mancloud-play-package.nix* contains your packaging code
with import <nixpkgs> {};
{
pidFile ? "/dev/null"
, devMode ? false
, extraConfig ? ""
}:
assert (builtins.isBool devMode);
let
localDevConfig = ./conf/local.conf;
generatedConfig = writeText "mancloud-play.conf" ''
${builtins.optionalString devMode "include \"local.dev.conf\""}
${extraConfig}
'';
in
stdenv.mkDerivation rec {
name= "mancloud-play";
src = ./src/play.mancloud.eu;
buildInputs = [ jre jdk sbt makeWrapper ];
configurePhase =
''
${builtins.optionalString devMode "cp ${localDevConfig}
conf/local.dev.conf"}
ln -s ${generatedConfig} conf/local.conf
sbt playUpdateSecret
'';
buildPhase = "sbt stage";
installPhase =
''
cp -r target/universal/stage $out
wrapProgram $out/bin/mancloud-play \
--set JAVA_HOME ${jre} \
--suffix PATH : "${gawk}/bin" \
--add-flags \
-Dpidfile.path=${pidFile}
mkdir -p $out/etc
cat >$out/etc/process_config <<EOF
container_process=$out/bin/mancloud-play
EOF
'';
dontStrip = true;
preFixup = "rm -rf $out/share/doc";
}
On Wed, Jul 6, 2016 at 5:10 PM 4levels <4levels at gmail.com> wrote:
> Hi Teo,
>
> I knew I was getting off course with my conclusions, thanks for clarifying
> this!
>
> I'll try to give you an overview, I don't mind adding you to our private
> Bitbucket repo if you'd like to see all files and folders.
> I still don't know where I should add the statements to have the play
> project deployed. All I have for now is the project's directory in a
> subfolder, src/play.mancloud.eu
>
> I still have many questions regarding nixo(p)s internals:
> - how does the sequence of the nixops modify files matter?
> - when to use with *import <nixpkgs>;* or *{ stdenv, lib, config, pkgs,
> ... }:* and what are the differences
> - ..
>
> I'll try to give you an explanation of how the deploy scripts are composed
> below. There's still a lot of room for improvements and regrouping of
> statements as I'm still a nix beginner..
>
>
> *nixops info* output
> Nix expressions: vultr.nix defaults-local.nix defaults.nix
> servers-local.nix keys-vm01.nix platform-local.nix
>
> *vultr.nix* contains Vultr specifics + the collectd setup
>
> {
> defaults = {
> deployment = {
> targetEnv = "none";
> };
> fileSystems."/" =
> {
> device = "/dev/vda1";
> options = [ "noatime" "nodiratime" "discard" ];
> };
> swapDevices = [
> {
> device = "/dev/vda2";
> }
> ];
> boot.initrd.availableKernelModules = [ "ata_piix" "uhci_hcd" "virtio_pci" "virtio_blk" ];
> boot.loader.grub.enable = true;
> boot.loader.grub.version = 2;
> boot.loader.grub.device = "/dev/vda";
> services.collectd.extraConfig = ''
> <Plugin df>
> Device "/dev/vda1"
> MountPoint "/"
> </Plugin>
> <Plugin interface>
> Interface "enp0s3"
> IgnoreSelected false
> </Plugin>
> '';
> };
> }
>
> *defaults-local.nix* contains some tweaks for my local vm as well as the
> copied call to parsets (which I renamed to mancloud-play)
>
> with import <nixpkgs>;
> {
> defaults =
> { stdenv, lib, config, pkgs, ... }:
> let
> wrapper = pkgs.callPackage ./mancloud-play-wrapper.nix {
> mancloud-play = import ./mancloud-play-package.nix;
> port = 9000;
> };
> in
> {
> programs.bash.promptInit = ''
> # Provide a nice prompt if the terminal supports it.
> if [ "$TERM" != "dumb" -o -n "$INSIDE_EMACS" ]; then
> PROMPT_COLOR="1;34m"
> let $UID && PROMPT_COLOR="1;34m"
> PS1="\n\[\033[$PROMPT_COLOR\][\u@\h:\w]\\$\[\033[0m\] "
> if test "$TERM" = "xterm"; then
> PS1="\[\033]3;\h:\u:\w\007\]$PS1"
> fi
> fi
> '';
> systemd.services = {
> inherit (wrapper.systemd.services) mancloud-play;
> };
> };
> }
>
> *defaults.nix* contains quite some services (key deployments, hostfile
> generation based on active projects, the mancloud (PHP) package, PHP-FPM
> tweaks and an alert service to monitor our queues, users, nginx, mysql,
> collectd, fail2ban, .. I've removed quite some config as this file is
> currently 540 lines long
>
> with import <nixpkgs/lib>;
> {
> defaults =
> { config, pkgs, lib, nodes, ... }:
> let
> serverKeys = keys:
> lib.genAttrs keys (n:
> {
> text = lib.removeSuffix "\n" (builtins.readFile (./keys + "/${builtins.replaceStrings ["@"] ["-"] n}") );
> group = "keys";
> permissions = "0640";
> }
> )
> ;
> ....
>
> in
> {
> deployment.keys = serverKeys [
> "mancloud.amazon.iam.key_id"
> "mancloud.amazon.iam.access_key"
> "mancloud.amazon.iam.passphrase"
> "mancloud._test.password"
> "phpmyadmin.password"
> "phpmyadmin.secret"
> ];
> environment.systemPackages = with pkgs; [
> wget
> unzip
> gitMinimal
> tmux
> mariadb
> php
> duplicity
> nodejs
> redis
> phpPackages.composer
> phpPackages.redis
> letsencrypt
> ];
> ....
> };
> }
>
> *servers-local.nix* contains the specific machine declarations (locally 1
> machine vm01) like ssl config, nfs mounts, libvirt related network config
> and some overrides from the defaults (stiil some reorganisation needed).
> It imports the extra service configuration (here diem-service.nix)
>
> with import <nixpkgs/lib>;
> let
> sslCert = ./src/ssl-cert.pem;
> sslKey = ./src/ssl-key.pem;
> sslCA = ./src/ssl-ca.pem;
> sslDHParam = ./src/ssl-dhparam.pem;
> nginxSslConfig = ''
> ${builtins.readFile ./src/nginx-ssl.conf}
> ssl_certificate ${sslCert};
> ssl_certificate_key ${sslKey};
> ssl_trusted_certificate ${sslCA};
> ssl_dhparam ${sslDHParam};
> '';
>
> in
> {
> vm01 =
> { config, pkgs, nodes, ... }:
> {
> imports = [ ./diem-service.nix ];
> deployment = {
> targetHost = "192.168.121.50";
> };
> environment.systemPackages = with pkgs; [
> strace
> gitAndTools.git-crypt
> ];
> services.mysql.enable = true;
> networking.hostName = "vm01"; # Define your hostname.
> networking.enableIPv6 = false;
> networking.extraHosts = "192.168.121.1 d01 d01.local";
> networking.interfaces = {
> enp0s3 = {
> ip4 = [ { address = "192.168.121.50"; prefixLength = 24; } ];
> };
> enp0s9 = {
> ip4 = [ { address = "192.168.0.98"; prefixLength = 24; } ];
> };
> };
> networking.nameservers = [ "192.168.121.1" ];
> networking.defaultGateway = "192.168.121.1";
> networking.firewall.allowedTCPPorts = [ 22 80 443 9000 ];
> fileSystems."/data/dev" = {
> device = "d01:/data/dev";
> fsType = "nfs";
> options = [ "defaults" "noatime" "nolock" "noacl" ];
> };
> services.collectd.enable = false;
> services.nginx = {
> httpConfig = ''
> # phpmyadmin
> server {
> listen 443 ssl spdy;
> server_name
> pma-local.mancloud.eu
> ;
> allow 192.168.121.1;
> allow 192.168.0.0/24;
> allow 192.168.1.0/24;
> allow 127.0.0.1;
> deny all;
> ${nginxSslConfig}
> root ${import ./phpmyadmin-package.nix};
> access_log '/tmp/pma-access.log';
> error_log '/tmp/pma-error.log' debug;
> location ~ "^(.+\.php)($|/)" {
> include ${pkgs.nginx}/conf/mime.types;
> ${builtins.readFile ./src/nginx-php-config.conf}
> }
> location / {
> include ${pkgs.nginx}/conf/mime.types;
> ${builtins.readFile ./src/nginx-rewrite.conf}
> }
> }
> '';
> };
> };
> }
>
>
> *diem-service.nix* contains the configuration for diem (PHP Symfony based
> CMF) and is quite large as well (+500 lines). I removed less relevant parts
>
> { config, lib, pkgs, nodes, ... }:
> let
> cfg = config.services.diem;
> serviceDir = "/var/www";
> systemdService = name: value:
> {
> name = "diem-${name}";
> value = {
> description = "Diem ${name} service";
> wantedBy = [ "multi-user.target" "nginx.target" ];
> after = [ "keys.target" "network.target" "mysql.target" ] ++
> lib.mapAttrsToList (n: v:
> "diem-${n}.service"
> ) (lib.filterAttrs (n: v: n < name) cfg.platforms);
> requires = [ "keys.target" "network.target" "mysql.target" ] ++
> lib.mapAttrsToList (n: v:
> "diem-${n}.service"
> ) (lib.filterAttrs (n: v: n < name) cfg.platforms);
> environment = {
> inherit (config.environment.variables) SSL_CERT_FILE;
> };
> serviceConfig.ExecStart = "${serviceDir}/${name}/nixSetup.sh";
> };
> };
> serverActivation = value:
> lib.concatStrings (lib.mapAttrsToList(n: v:
> if (lib.isAttrs(v) && lib.hasAttr("platform") v) then
> "${packageActivation n v};"
> else ""
> ) value
> );
> packageActivation = name: value:
> {
> name = "diem-${name}";
> value =
> ''
> # create / symlink project dirs
> mkdir -p ${serviceDir}/${name}
> mkdir -p
> ${serviceDir}/${name}/{cache/dm,config/dm,data/backup/db,data/backup/uploads,data/restore/db,data/restore/uploads,data/dm/i18n,data/exports,log,web/uploads}
>
> # letsEncrypt script
> cp ${pkgs.writeText "letsEncrypt.sh" "${letsEncrypt name value}"
> } ${serviceDir}/${name}/letsEncrypt.sh
> chmod +x ${serviceDir}/${name}/letsEncrypt.sh
>
> # nixSetup script
> cp ${pkgs.writeText "nixSetup.sh" "${nixSetup name value}" }
> ${serviceDir}/${name}/nixSetup.sh
> chmod +x ${serviceDir}/${name}/nixSetup.sh
>
> # s3Backup script
> cp ${pkgs.writeText "s3Backup.sh" "${s3Backup name}" }
> ${serviceDir}/${name}/s3Backup.sh
> chmod +x ${serviceDir}/${name}/s3Backup.sh
> '';
> };
> diem = (import ./diem-package.nix);
> ...
>
> in
> with lib;
> {
> options = {
> services.diem = {
> platforms = mkOption {
> default = {};
> example = {
> test = {
> database = {
> password = "foopass";
> };
> timezone = "Europe/Brussels";
> };
> };
> };
> };
> };
>
> config = mkIf (cfg.platforms != {}) {
> system.activationScripts = mapAttrs' packageActivation cfg.platforms;
> systemd.services = mapAttrs' systemdService cfg.platforms // timers
> cfg.platforms;
> };
> }
>
> *diem-package.nix* contains the packaging statements, read from a github
> repo
>
> with import <nixpkgs> {};
> pkgs.stdenv.mkDerivation rec {
> name = "diem-1.0.0";
> src = pkgs.fetchgit {
> url = "https://github.com/diem-project/diem.git";
> rev = "refs/heads/master";
> sha256 = "11scd9z7h91bd242gvy0grnlx75d25ckx1k0k3qvz74p55f1kww7";
> };
>
> buildPhase = "true";
> installPhase =
> ''
> mkdir -p $out
> cp -r * $out
> '';
> }
>
>
>
> *keys-vm01.nix* contains the inclusions of the configuration keys and
> other sensitive data for this host
>
> {
> vm01 =
> { config, pkgs, lib, ... }:
> let
> serverKeys = keys:
> lib.genAttrs keys (n:
> {
> text = lib.removeSuffix "\n" (builtins.readFile (./keys/vm01 + "/${builtins.replaceStrings ["@"] ["-"] n}") );
> group = "keys";
> permissions = "0640";
> }
> )
> ;
> in
> {
> deployment.keys = serverKeys [
> "diem.project.database.password"
> "diem.project.encryption.cipher"
> "diem.project.encryption.key"
> ...
> ];
> };
> }
>
>
> platform-local.nix contains the project definitions per server
>
> with import <nixpkgs/lib>;
> {
> vm01 =
> { config, pkgs, ... }:
> {
> services.diem.platforms = {
> project = {
> domain = "local.project";
> path = "/data/dev/projects/project";
> };
> };
> ...
> };
> }
>
>
> Kind regards and thank you again for your willing and friendly attitude!
>
> Erik
>
>
>
>
>
> On Wed, Jul 6, 2016 at 5:41 AM Teo Klestrup Röijezon <teo at nullable.se>
> wrote:
>
>> HI Erik,
>>
>> That's pretty much entirely wrong. :P ParseTS is just a linter script for
>> the game scripting language TorqueScript. ParseTS-Playground was a pastebin
>> that would run the submitted code through the linter. For example, see
>> https://parsets-playground.nullable.se/snippets/13. The datastore used
>> was PostgreSQL.
>>
>> Anyway, apart from the ParseTS stuff, at least those scripts should be
>> pretty much straightforward to copy to any Play application, though for the
>> config stuff to work you'll need to add the line 'include "local.conf"' to
>> your conf/application.conf.
>>
>> Any chance you could post your current setup and the errors you get?
>>
>> // Teo
>>
>> On 6 July 2016 at 04:31, 4levels <4levels at gmail.com> wrote:
>>
>>> Hi Teo,
>>>
>>> I've come quite far in setting up things, but I keep running into
>>> building errors.
>>> It has everything to do with me removing all references to parsets and
>>> postgres and renaming things here and there, trying to merge them with the
>>> current deploy setup.
>>>
>>> Do I understand correctly that parsets is a library to store data, using
>>> postgres in the background? I'd like to start using Event Sourcing with
>>> Scala / Akka so I don't need a datastore like parsets, correct? I'm very
>>> unsure about this as I literally started today with learning Scala / Play.
>>> I got my toes wet with Java before but that's really it.
>>>
>>> Something else I found interesting as I'm quite an Nginx fan and have
>>> nginx running with proxies already: Nginx has capabilities to deal with
>>> Java in different ways, as proxy or tied with eg Clojure for even faster
>>> results..
>>>
>>> The journey continues ;-)
>>>
>>>
>>> Kind regards,
>>>
>>> Erik
>>>
>>> On Tue, Jul 5, 2016 at 10:23 PM 4levels <4levels at gmail.com> wrote:
>>>
>>>> Hi Teo,
>>>>
>>>> Thank you for your explanation and quick qualitative response!
>>>>
>>>> I'll be looking at your code asap and report back with my experiences
>>>> ;-)
>>>>
>>>> Kind regards,
>>>>
>>>> Erik
>>>>
>>>> On Tue, Jul 5, 2016, 22:08 Teo Klestrup Röijezon <teo at nullable.se>
>>>> wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> A JRE should be enough for running it, but you need sbt and a JDK for
>>>>> building. I've got a derivation for a Play website at
>>>>> https://github.com/BlocklandGlass/ParseTS-Playground/blob/master/parsets-playground.nix,
>>>>> with the NixOS/NixOps setup at
>>>>> https://github.com/BlocklandGlass/ParseTS-Playground/tree/master/deployment
>>>>> .
>>>>>
>>>>> The gist of it is to run "sbt stage" in the build phase, and to then
>>>>> take "target/universal/stage" as your build output. However, you'll also
>>>>> need to wrap the launcher script to add your JRE and to add gawk (which the
>>>>> launcher script requires). Finally, on any modern system (such as NixOS)
>>>>> you'll also want to disable Play's PID file management, since systemd takes
>>>>> care of that anyway. I didn't in that script, but you'll probably also want
>>>>> to add a testing phase as part of the build.
>>>>>
>>>>> The big drawback with this approach is that SBT downloads all
>>>>> dependencies from the internet on demand, which won't work on a Nix setup
>>>>> with proper isolation (ideally, builds should only have network access if
>>>>> they deterministically produce a given hash).
>>>>>
>>>>> I've been toying with the idea of writing a sbt2nix SBT plugin that
>>>>> generates Nix definitions to build a local maven mirror for the
>>>>> dependencies, but I haven't got around to that (yet).
>>>>>
>>>>> // Teo
>>>>>
>>>>> On 5 July 2016 at 21:52, 4levels <4levels at gmail.com> wrote:
>>>>>
>>>>>> Hi Nix-devs,
>>>>>>
>>>>>> This is a plain request for assistance / best practices for using
>>>>>> Nixos with Java / Scala / Play. Akka with EventSourcing are also a topic
>>>>>> of interest.
>>>>>>
>>>>>> I'm currently trying to get a Scala Play app up and running on my
>>>>>> nixOps deployed machines. As I'm very unfamiliar with running Java based
>>>>>> apps, I'd like to know if someone has experience on the common pitfalls and
>>>>>> tips on keeping the servers healthy (I just caused my laptop's 8 cores to
>>>>>> go 100% without being able to stop the server started by the activator
>>>>>> call).
>>>>>>
>>>>>> I've seen some related packages in nixpkgs and have many questions
>>>>>> like eg. do I need sbt (which seems to provide typesafe - activator) and a
>>>>>> jdk on the production servers or are is a jre sufficient? How do I deploy
>>>>>> and run a Java app developed locally?
>>>>>> And how do I set-up a local nixos vm for Java development?
>>>>>>
>>>>>> I'm still investigating and learning a lot myself, so nix-related
>>>>>> knowledge is my main concern here (as I need to figure out the rest myself
>>>>>> anyway ;-)
>>>>>>
>>>>>> I'll be happy to share my findings and configuration / setup..
>>>>>>
>>>>>>
>>>>>> Kind regards,
>>>>>>
>>>>>> Erik
>>>>>>
>>>>>> _______________________________________________
>>>>>> nix-dev mailing list
>>>>>> nix-dev at lists.science.uu.nl
>>>>>> http://lists.science.uu.nl/mailman/listinfo/nix-dev
>>>>>>
>>>>>>
>>>>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.science.uu.nl/pipermail/nix-dev/attachments/20160706/75829832/attachment-0001.html>
More information about the nix-dev
mailing list