<?xml version="1.0" encoding="utf-8"?><feed xmlns="http://www.w3.org/2005/Atom" ><generator uri="https://jekyllrb.com/" version="3.10.0">Jekyll</generator><link href="https://blog.kmassada.com/feed.xml" rel="self" type="application/atom+xml" /><link href="https://blog.kmassada.com/" rel="alternate" type="text/html" /><updated>2026-03-02T23:58:52+00:00</updated><id>https://blog.kmassada.com/feed.xml</id><title type="html">kmassada blog</title><subtitle>Just another tech blog.</subtitle><author><name>Kenneth Massada</name></author><entry><title type="html">Wireguard Server on GCP with Gl-iNet Beryl as a client</title><link href="https://blog.kmassada.com/quickstart-wireguard-beryl-gcp/" rel="alternate" type="text/html" title="Wireguard Server on GCP with Gl-iNet Beryl as a client" /><published>2021-08-01T04:00:00+00:00</published><updated>2021-08-01T04:00:00+00:00</updated><id>https://blog.kmassada.com/quickstart-wireguard-beryl-gcp</id><content type="html" xml:base="https://blog.kmassada.com/quickstart-wireguard-beryl-gcp/"><![CDATA[<h2 id="setup-server">Setup Server</h2>

<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c"># reserve public IP</span>
gcloud compute addresses create wireguard <span class="se">\</span>
    <span class="nt">--region</span> us-west <span class="se">\</span>
    <span class="nt">--ip-version</span> IPV4

<span class="c"># save IP</span>
<span class="nv">IP_ADDRESS</span><span class="o">=</span>

<span class="c"># create firewall rule to allow 51820</span>
gcloud compute firewall-rules create allow-wireguard <span class="se">\</span>
    <span class="nt">--network</span> default <span class="se">\</span>
    <span class="nt">--action</span> allow <span class="se">\</span>
    <span class="nt">--direction</span> ingress <span class="se">\</span>
    <span class="nt">--rules</span> udp:51820 <span class="se">\</span>
    <span class="nt">--source-ranges</span> 0.0.0.0/0

<span class="c"># create instance with the public IP assigned</span>
gcloud compute instances create wireguard <span class="se">\</span>
    <span class="nt">--address</span><span class="o">=</span><span class="nv">$IP_ADDRESS</span> <span class="se">\</span>
    <span class="nt">--network</span> default <span class="se">\</span>
    <span class="nt">--machine-type</span><span class="o">=</span>f1-micro <span class="se">\</span>
    <span class="nt">--image-family</span><span class="o">=</span>debian-10 <span class="se">\</span>
    <span class="nt">--image-project</span><span class="o">=</span>debian-cloud <span class="se">\</span>
    <span class="nt">--can-ip-forward</span> <span class="se">\</span>
    <span class="nt">--boot-disk-size</span><span class="o">=</span>10GB <span class="se">\</span>
    <span class="nt">--zone</span><span class="o">=</span>us-west1-a

<span class="c">#ssh into the server</span>
gcloud  compute ssh <span class="nt">--zone</span><span class="o">=</span>us-west1-a wireguard
</code></pre></div></div>

<h2 id="pre-reqs">Pre-Reqs</h2>

<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nb">sudo </span>sh <span class="nt">-c</span> <span class="s2">"echo 'deb http://deb.debian.org/debian buster-backports main' &gt;&gt; /etc/apt/sources.list.d/backports.list"</span>
<span class="nb">sudo </span>sh <span class="nt">-c</span> <span class="s2">"printf 'Package: *</span><span class="se">\n</span><span class="s2">Pin: release a=buster-backports</span><span class="se">\n</span><span class="s2">Pin-Priority: 90</span><span class="se">\n</span><span class="s2">' &gt;&gt; /etc/apt/preferences.d/limit-backports"</span>
<span class="nb">sudo </span>apt update
<span class="nb">sudo </span>apt <span class="nb">install </span>wireguard <span class="nt">--assume-yes</span>
<span class="nb">sudo </span>apt <span class="nb">install </span>linux-headers-<span class="si">$(</span><span class="nb">uname</span> <span class="nt">-r</span><span class="si">)</span>

<span class="nb">sudo </span>apt update <span class="o">&amp;&amp;</span> <span class="nb">sudo </span>apt upgrade

<span class="nb">sudo </span>vi /etc/sysctl.conf
<span class="nb">sudo </span>sysctl <span class="nt">-p</span> /etc/sysctl.conf
<span class="nb">sudo </span>sysctl <span class="nt">-w</span> net.ipv4.ip_forward<span class="o">=</span>1

<span class="nb">sudo </span>reboot
</code></pre></div></div>

<h2 id="configure-server">Configure Server</h2>

<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c"># Generate server private and public keys</span>
<span class="o">(</span><span class="nb">umask </span>077 <span class="o">&amp;&amp;</span> wg genkey <span class="o">&gt;</span> wg-private.key<span class="o">)</span>
wg pubkey &lt; wg-private.key <span class="o">&gt;</span> wg-public.key

<span class="c">## Print Private Key Needed in /etc/wireguard/wg0.conf</span>
<span class="nb">cat </span>wg-private.key

<span class="c"># find the name of interface ()</span>
ip <span class="nt">-o</span> <span class="nt">-4</span> route show to default | <span class="nb">awk</span> <span class="s1">'{print $5}'</span>
</code></pre></div></div>

<p>Notable configs to keep in mind</p>

<ol>
  <li>Address of the server is the range of the IPs that live in the vpn subnet, mine for instance is <code class="language-plaintext highlighter-rouge">10.0.2.1/24</code>, I’ve confirmed it using 3 sources, <a href="https://www.wireguard.com/quickstart/">wireguard quickstart guide</a>, <a href="https://wireguard.how/server/google-cloud-platform/">wireguard.how guide</a> and <a href="https://serversideup.net/courses/gain-flexibility-and-increase-privacy-with-wireguard-vpn/">serversideup guide</a></li>
  <li>Saveconfig is important for my guide I use it to add peers. It allows you to run commands at the cli and save them to the <code class="language-plaintext highlighter-rouge">wg0.conf</code></li>
  <li>
    <YOUR_NETWORK_INTERFACE> is the name of the interface where the traffic is NATted. 

</YOUR_NETWORK_INTERFACE>
  </li>
</ol>
<div class="language-conf highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c"># filename: /etc/wireguard/wg0.conf
# define the WireGuard service
</span>[<span class="n">Interface</span>]

<span class="c"># contents of file wg-private.key that was recently created
</span><span class="n">PrivateKey</span> = &lt;<span class="n">SERVER_PRIVATE_KEY</span>&gt;

<span class="c"># MTU
</span><span class="n">MTU</span> = <span class="m">1380</span>

<span class="c"># Addy of server
</span><span class="n">Address</span> = <span class="m">10</span>.<span class="m">0</span>.<span class="m">2</span>.<span class="m">1</span>/<span class="m">24</span>

<span class="c"># NAT
</span><span class="n">PostUp</span> = <span class="n">iptables</span> -<span class="n">A</span> <span class="n">FORWARD</span> -<span class="n">i</span> %<span class="n">i</span> -<span class="n">j</span> <span class="n">ACCEPT</span>; <span class="n">iptables</span> -<span class="n">t</span> <span class="n">nat</span> -<span class="n">A</span> <span class="n">POSTROUTING</span> -<span class="n">o</span> &lt;<span class="n">YOUR_NETWORK_INTERFACE</span>&gt; -<span class="n">j</span> <span class="n">MASQUERADE</span>
<span class="n">PostDown</span> = <span class="n">iptables</span> -<span class="n">D</span> <span class="n">FORWARD</span> -<span class="n">i</span> %<span class="n">i</span> -<span class="n">j</span> <span class="n">ACCEPT</span>; <span class="n">iptables</span> -<span class="n">t</span> <span class="n">nat</span> -<span class="n">D</span> <span class="n">POSTROUTING</span> -<span class="n">o</span> &lt;<span class="n">YOUR_NETWORK_INTERFACE</span>&gt; -<span class="n">j</span> <span class="n">MASQUERADE</span>

<span class="c"># UDP service port; 51820 is a common choice for WireGuard
</span><span class="n">ListenPort</span> = <span class="m">51820</span>

<span class="c"># allow changes from cli to be added
</span><span class="n">SaveConfig</span> = <span class="n">True</span>
</code></pre></div></div>

<p>Now let’s bootstrap the interface and enable it to start on reboot.</p>

<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nb">sudo </span>wg-quick up wg0
<span class="nb">sudo </span>systemctl <span class="nb">enable </span>wg-quick@wg0
</code></pre></div></div>

<h3 id="verify-server-setup">Verify Server Setup</h3>

<p>Now let’s verify before we move on.</p>

<ol>
  <li>the values of these two must match</li>
</ol>

<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">$ </span><span class="nb">sudo grep</span> <span class="nt">-i</span> private  /etc/wireguard/wg0.conf
&lt;SERVER_PRIVATE_KEY&gt;
<span class="nv">$ </span><span class="nb">cat </span>wg-private.key
&lt;SERVER_PRIVATE_KEY&gt;
</code></pre></div></div>

<ol>
  <li>the following values for public key must match</li>
</ol>

<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">$ </span><span class="nb">sudo </span>wg show wg0
interface: wg0
  public key: &lt;SERVER_PUBLIC_KEY&gt;
  private key: <span class="o">(</span>hidden<span class="o">)</span>
  listening port: 51820
<span class="nv">$ </span><span class="nb">cat </span>wg-public.key
&lt;SERVER_PUBLIC_KEY&gt;
</code></pre></div></div>

<ol>
  <li>IP address must be check out</li>
</ol>

<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">$ </span>ip address show dev wg0
9: wg0: &lt;POINTOPOINT,NOARP,UP,LOWER_UP&gt; mtu 1380 qdisc noqueue state UNKNOWN group default qlen 1000
    <span class="nb">link</span>/none
    inet 10.0.2.1/24 scope global wg0
       valid_lft forever preferred_lft forever
<span class="nv">$ </span><span class="nb">sudo grep</span> <span class="nt">-i</span> address /etc/wireguard/wg0.conf
Address <span class="o">=</span> 10.0.2.1/24
</code></pre></div></div>

<ol>
  <li>Verify the steps for when the interface comes online, watching for iptables routes, mtu, and address set correctly</li>
</ol>

<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">$ </span><span class="nb">sudo </span>wg-quick up wg0
<span class="o">[</span><span class="c">#] ip link add wg0 type wireguard</span>
<span class="o">[</span><span class="c">#] wg setconf wg0 /dev/fd/63</span>
<span class="o">[</span><span class="c">#] ip -4 address add 10.0.2.1/24 dev wg0</span>
<span class="o">[</span><span class="c">#] ip link set mtu 1380 up dev wg0</span>
<span class="o">[</span><span class="c">#] iptables -A FORWARD -i wg0 -j ACCEPT; iptables -t nat -A POSTROUTING -o ens4 -j MASQUERADE</span>
</code></pre></div></div>

<h2 id="configure-client">Configure Client</h2>

<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c"># Generate client private and public keys</span>
<span class="nb">sudo </span>wg show wg0
<span class="o">(</span><span class="nb">umask </span>077 <span class="o">&amp;&amp;</span> wg genkey <span class="o">&gt;</span> wg-private-client.key<span class="o">)</span>
wg pubkey &lt; wg-private-client.key <span class="o">&gt;</span> wg-public-client.key

<span class="c">## Print Private Key Needed in ~/wg-client.conf</span>
<span class="nb">cat </span>wg-private-client.key
</code></pre></div></div>

<p>let’s work on the config, in the GLi Admin panel, under wireguard client, create a new configuration file with the following valiables</p>

<p>NOTE: config is picky, remove all the comments and make sure there’s spacing between the equal sign, the variable name and the value. <a href="https://docs.gl-inet.com/en/3/app/wireguard/#input-the-configuration">gl-iNet’s wireguard guide</a> is comprehensive for this step.</p>

<div class="language-conf highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c"># filename: ~/wg-client.conf
# define the local WireGuard interface (client)
</span>[<span class="n">Interface</span>]

<span class="c"># contents of wg-private-client.key
</span><span class="n">PrivateKey</span> = &lt;<span class="n">CLIENT</span> <span class="n">PRIVATE_KEY</span>&gt;

<span class="c"># the IP address of this client on the WireGuard network
</span><span class="n">Address</span> = <span class="m">10</span>.<span class="m">0</span>.<span class="m">2</span>.<span class="m">2</span>/<span class="m">32</span>

<span class="c"># DNS
</span><span class="n">DNS</span> = <span class="m">1</span>.<span class="m">1</span>.<span class="m">1</span>.<span class="m">1</span>

<span class="c"># MTU
</span><span class="n">MTU</span> = <span class="m">1460</span>

<span class="c"># define the remote WireGuard interface (server)
</span>[<span class="n">Peer</span>]

<span class="c"># from `sudo wg show wg0`
</span><span class="n">PublicKey</span> = &lt;<span class="n">SERVER</span> <span class="n">PUBLIC_KEY</span>&gt;

<span class="c"># the IP address of the server on the WireGuard network 
</span><span class="n">AllowedIPs</span> = <span class="m">0</span>.<span class="m">0</span>.<span class="m">0</span>.<span class="m">0</span>/<span class="m">0</span>

<span class="c"># public IP address and port of the WireGuard server
</span><span class="n">Endpoint</span> = <span class="n">XX</span>.<span class="n">XX</span>.<span class="n">XX</span>.<span class="n">XX</span>:<span class="m">51820</span>
</code></pre></div></div>

<p>Now let’s write into the server config the peer config for the client.</p>

<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nb">sudo </span>wg <span class="nb">set </span>wg0 peer &lt;PUBLIC CLIENT KEY&gt; allowed-ips 10.0.2.2
</code></pre></div></div>

<h2 id="verify-client-config">Verify client config</h2>

<p>Now let’s verify</p>

<ol>
  <li>public client key should now be part of the wg0 config. the allowed IPs should show the single IP range of the client, in this case <code class="language-plaintext highlighter-rouge">10.0.2.2/32</code>.</li>
</ol>

<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">$ </span><span class="nb">cat </span>wg-public-client.key
 &lt;CLIENT PUBLIC_KEY&gt;
<span class="nv">$ </span><span class="nb">sudo </span>wg show wg0
interface: wg0
  public key: &lt;SERVER PUBLIC_KEY&gt;
  private key: <span class="o">(</span>hidden<span class="o">)</span>
  listening port: 51820

peer: &lt;CLIENT PUBLIC_KEY&gt;
  endpoint: XX.XX.XX.XX:40253
  allowed ips: 10.0.2.2/32
</code></pre></div></div>

<ol>
  <li>you’ve validated that on the GL-iNet side, the following values in the config are the Client’s Private Key and the Server’s Public key.</li>
</ol>

<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code>PrivateKey <span class="o">=</span> &lt;CLIENT PRIVATE_KEY&gt;
PublicKey <span class="o">=</span> &lt;SERVER PUBLIC_KEY&gt;
</code></pre></div></div>

<h2 id="connect">Connect</h2>

<p>In order to connect, you simply go to the Wireguard client and establish connection. <a href="https://docs.gl-inet.com/en/3/app/wireguard">gl-iNet’s wireguard guide</a> is comprehensive for this step.</p>

<p>Things to remember once the client’s connection is established</p>

<ol>
  <li>when you’ve established connection on the GL-iNet side it shows the IP address you’ve told your server the client would have.</li>
</ol>

<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code>IP Address10.0.2.2
Upload / Download82.41 MB / 90.19 MB
</code></pre></div></div>

<ol>
  <li>This command will show you data transfer rates, and the client’s endpoint</li>
</ol>

<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">$ </span><span class="nb">sudo </span>wg show wg0
...
peer: &lt;CLIENT PUBLIC_KEY&gt;
  endpoint: XX.XX.XX.XX:XXX
  allowed ips: 10.0.2.2/32
..CONNECTION INFO..
</code></pre></div></div>

<h2 id="troublehsoot">Troublehsoot</h2>

<h3 id="making-changes">Making changes</h3>

<ol>
  <li>edit conf
    <div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nb">sudo </span>vi /etc/wireguard/wg0.conf
</code></pre></div>    </div>
  </li>
  <li>down interface
    <div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nb">sudo </span>wg-quick down wg0
</code></pre></div>    </div>
  </li>
  <li>sysctl to up interface
    <div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nb">sudo </span>systemctl restart wg-quick@wg0.service
</code></pre></div>    </div>
  </li>
  <li>check if changes are live
    <div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code>ug 03 23:25:57 wireguard wg-quick[1004]: <span class="o">[</span><span class="c">#] ip link add wg0 type wireguard</span>
Aug 03 23:25:57 wireguard wg-quick[1004]: <span class="o">[</span><span class="c">#] wg setconf wg0 /dev/fd/63</span>
Aug 03 23:25:57 wireguard wg-quick[1004]: <span class="o">[</span><span class="c">#] ip -4 address add 10.0.2.1/24 dev wg0</span>
Aug 03 23:25:57 wireguard wg-quick[1004]: <span class="o">[</span><span class="c">#] ip link set mtu 1360 up dev wg0</span>
Aug 03 23:25:57 wireguard wg-quick[1004]: <span class="o">[</span><span class="c">#] iptables -A FORWARD -i wg0 -j ACCEPT; iptables -t nat -A POSTROUTING </span>
Aug 03 23:25:57 wireguard systemd[1]: Started WireGuard via wg-quick<span class="o">(</span>8<span class="o">)</span> <span class="k">for </span>wg0.
</code></pre></div>    </div>
  </li>
  <li>show interface
    <div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code>ip address show dev wg0
</code></pre></div>    </div>
  </li>
  <li>show status
    <div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nb">sudo </span>wg show wg0
</code></pre></div>    </div>
  </li>
</ol>

<h3 id="gcp-troubleshoot">GCP Troubleshoot</h3>

<ol>
  <li>
    <p>it is very important that the instance <code class="language-plaintext highlighter-rouge">--can-ip-forward</code>, and that the Natting and Masquerade rules are set.</p>
  </li>
  <li>
    <p>GCP’s firewall rules allows to enable logging, to log when you</p>
  </li>
</ol>

<p>this Cloud Logging query</p>

<div class="language-s highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">logName</span><span class="o">:</span><span class="p">(</span><span class="n">projects</span><span class="o">/&lt;</span><span class="n">YOUR</span><span class="w"> </span><span class="n">PROJECT</span><span class="o">&gt;/</span><span class="n">logs</span><span class="o">/</span><span class="n">compute.googleapis.com</span><span class="o">%</span><span class="m">2</span><span class="n">Ffirewall</span><span class="p">)</span><span class="w"> </span><span class="n">AND</span><span class="w"> </span><span class="n">jsonPayload.rule_details.reference</span><span class="o">:</span><span class="p">(</span><span class="s2">"network:default/firewall:allow-wireguard"</span><span class="p">)</span><span class="w">
</span></code></pre></div></div>

<p>allowed me to confirm traffic/connection attempts are reaching my node.</p>

<div class="language-json highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="err">connection:</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="err">dest_ip:</span><span class="w"> </span><span class="s2">"10.138.0.3"</span><span class="w">
</span><span class="err">dest_port:</span><span class="w"> </span><span class="mi">51820</span><span class="w">
</span><span class="err">protocol:</span><span class="w"> </span><span class="mi">17</span><span class="w">
</span><span class="err">src_ip:</span><span class="w"> </span><span class="s2">"xx.xx.xx.xx"</span><span class="w">
</span><span class="err">src_port:</span><span class="w"> </span><span class="err">xxxxx</span><span class="w">
</span><span class="p">}</span><span class="w">
</span></code></pre></div></div>

<ol>
  <li>
    <p>GCP instances MTU, I’ve set the server side to 1360 and the client to 1480. this actually helped me increase my performance on the instance.</p>
  </li>
  <li>
    <p>This is a quick way to verify that you static IP was indeed attached to the node</p>
  </li>
</ol>

<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code>curl zx2c4.com/ip
xx.xx.xx.xx
</code></pre></div></div>

<h2 id="linux--nettools-troubleshooting">Linux / NetTools Troubleshooting</h2>

<p>once we can confirm traffic is reaching the vm’s nic, we can run this to be sure</p>

<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">$ </span><span class="nb">sudo </span>tcpdump <span class="nt">-i</span> YOUR INTERFACE NAME dst port 51820
...
04:14:28.618587 IP c-ISP-STUFF-EH.40253 <span class="o">&gt;</span> wireguard.us-west1-a.c.YOUR-PROJECT-EH.internal.51820: UDP, length 80
04:14:29.326714 IP c-ISP-STUFF-EH.40253 <span class="o">&gt;</span> wireguard.us-west1-a.c.YOUR-PROJECT-EH.internal.51820: UDP, length 80
04:14:29.394530 IP c-ISP-STUFF-EH.40253 <span class="o">&gt;</span> wireguard.us-west1-a.c.YOUR-PROJECT-EH.internal.51820: UDP, length 80
47 packets captured
47 packets received by filter
0 packets dropped by kernel
</code></pre></div></div>

<p>wecan also confirm pings from client to server</p>

<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">$ </span><span class="nb">sudo </span>tcpdump <span class="nt">-i</span> wg0 icmp
04:24:48.700291 IP 10.0.2.2 <span class="o">&gt;</span> 10.0.2.1: ICMP <span class="nb">echo </span>request, <span class="nb">id </span>48775, <span class="nb">seq </span>0, length 64
04:24:48.700364 IP 10.0.2.1 <span class="o">&gt;</span> 10.0.2.2: ICMP <span class="nb">echo </span>reply, <span class="nb">id </span>48775, <span class="nb">seq </span>0, length 64
</code></pre></div></div>

<p>wecan also troubleshoot dns lookups by sending an nlookup on my client</p>

<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code>04:29:12.956849 IP 10.0.2.2.54136 <span class="o">&gt;</span> one.one.one.one.domain: 56514+ A? google.com. <span class="o">(</span>28<span class="o">)</span>
04:29:12.965835 IP one.one.one.one.domain <span class="o">&gt;</span> 10.0.2.2.54136: 56514 1/0/0 A 142.250.217.110 <span class="o">(</span>44<span class="o">)</span>
</code></pre></div></div>

<p>We can similarly check for IP Leaks. As expected region was us-west1 and we expect the location to not be my current location but where the IP was registered.</p>

<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">$ </span>curl https://ipleak.net/json/
<span class="o">{</span>
    <span class="s2">"country_code"</span>: <span class="s2">"US"</span>,
    <span class="s2">"country_name"</span>: <span class="s2">"United States"</span>,
    <span class="s2">"region_code"</span>: <span class="s2">"OR"</span>,
    <span class="s2">"region_name"</span>: <span class="s2">"Oregon"</span>,
    <span class="s2">"continent_code"</span>: <span class="s2">"NA"</span>,
    <span class="s2">"continent_name"</span>: <span class="s2">"North America"</span>,
    <span class="s2">"city_name"</span>: <span class="s2">"The Dalles"</span>,
    <span class="s2">"postal_code"</span>: null,
    <span class="s2">"postal_confidence"</span>: null,
    <span class="s2">"latitude"</span>: 45.5999,
    <span class="s2">"longitude"</span>: <span class="nt">-121</span>.1871,
    <span class="s2">"accuracy_radius"</span>: 1000,
    <span class="s2">"time_zone"</span>: <span class="s2">"America</span><span class="se">\/</span><span class="s2">Los_Angeles"</span>,
    <span class="s2">"metro_code"</span>: 820,
    <span class="s2">"level"</span>: <span class="s2">"min"</span>,
    <span class="s2">"cache"</span>: 1628053227,
    <span class="s2">"ip"</span>: <span class="s2">"xx.xx.xx.xx"</span>,
    <span class="s2">"reverse"</span>: <span class="s2">""</span>,
    <span class="s2">"query_text"</span>: <span class="s2">"xx.xx.xx.xx"</span>,
    <span class="s2">"query_type"</span>: <span class="s2">"myip"</span>,
    <span class="s2">"query_date"</span>: 1628053227
<span class="o">}</span>
</code></pre></div></div>]]></content><author><name>Kenneth Massada</name></author><category term="gcp" /><category term="blog" /><category term="gcp" /><summary type="html"><![CDATA[Setting up wireguard server on gcp instance and connecting to it using my beryl travel router]]></summary></entry><entry><title type="html">Ghost Maintenance and Backup on GCS</title><link href="https://blog.kmassada.com/ghost-backup/" rel="alternate" type="text/html" title="Ghost Maintenance and Backup on GCS" /><published>2021-04-27T04:00:00+00:00</published><updated>2021-04-27T04:00:00+00:00</updated><id>https://blog.kmassada.com/ghost-backup</id><content type="html" xml:base="https://blog.kmassada.com/ghost-backup/"><![CDATA[<p>I’ve been using the blog platform ghost for hosting tadbit.cc, and now twdspodcast.com. Migrating to Ghost 4, this post is explores Ghost maintenance and backups. This article isn’t really useful if you aren’t familiar with my setup. <a href="/running-ghost-on-gcp">Running Ghost on GCP</a> and <a href="/lets-encrypt-nginx-docker-ghost">Running ghost blog with nginx reverse proxy and let’s encrypt</a>.</p>

<p>The TLDR is I build a gce instance that runs 2 containers, one for ghost and one for nginx. I use nginx as a reverse proxy to the ghost container. Recently I’ve added the complexity of saving the ghost data in mysql instead of a volume attached to the ghost container. Also, I now use let’s encrypt to generate or manage my certificates.</p>

<p>Few important assumptions to note for this post</p>

<ul>
  <li>I created my instance in <code class="language-plaintext highlighter-rouge">us-west1-a</code> and I created my storage bucket in <code class="language-plaintext highlighter-rouge">us-west1</code> to reduce cost.</li>
  <li>I created my instance with a service account, that service account and it’s scope is what allows me to backup to GCS without having to download keys.</li>
</ul>

<h2 id="backup">Backup</h2>

<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">now</span><span class="o">=</span><span class="si">$(</span><span class="nb">date</span> +<span class="s1">'%Y-%m-%d_%H-%M'</span><span class="si">)</span>
<span class="nb">mkdir</span> <span class="nt">-p</span> /tmp/data/<span class="nv">$now</span>
</code></pre></div></div>

<p>when we created the blog, we saved the blog files at <code class="language-plaintext highlighter-rouge">$data_path/blog</code>, we’ll tar the relevant files in that folder. I do use <code class="language-plaintext highlighter-rouge">data/redirect.json</code> and <code class="language-plaintext highlighter-rouge">settings/routes.yaml</code> quite a bit, and I have purchased 2 themes <code class="language-plaintext highlighter-rouge">{Massively-master,hue}</code>, this should exclude <code class="language-plaintext highlighter-rouge">logs</code>, <code class="language-plaintext highlighter-rouge">themes/casper</code></p>

<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nb">cd</span> <span class="nv">$data_path</span>
<span class="nb">tar</span> <span class="nt">-zcvf</span> /tmp/data/<span class="nv">$now</span>/blog-<span class="nv">$now</span>.tar.gz blog/images blog/data blog/settings blog/themes/<span class="o">{</span>Massively-master,hue<span class="o">}</span>
</code></pre></div></div>

<p>when we launched the mysql container, we used <strong>mysql</strong> as the name, therefore <code class="language-plaintext highlighter-rouge">docker exec mysql</code> will exec into our container</p>

<p>Note <code class="language-plaintext highlighter-rouge">ghost_sitename</code> is the name of the table. In we picked <code class="language-plaintext highlighter-rouge">${domain_short}_ghost</code> as a convention for our db names.</p>

<p>this can be verified by running</p>

<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code> docker <span class="nb">exec</span> <span class="nt">-t</span> mysql sh <span class="nt">-c</span> <span class="s1">'mysql -uroot -p"$MYSQL_ROOT_PASSWORD" -e "SHOW DATABASES;"'</span>
</code></pre></div></div>

<p>Now we actually backup the data</p>

<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code>docker <span class="nb">exec </span>mysql sh <span class="nt">-c</span> <span class="s1">'exec mysqldump ghost_sitename -uroot -p"$MYSQL_ROOT_PASSWORD"'</span> <span class="o">&gt;</span> /tmp/data/<span class="nv">$now</span>/ghost_sitename-<span class="nv">$now</span>.sql
</code></pre></div></div>

<p>After backing up I have 2 files I can use.</p>

<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code>/tmp/data <span class="nv">$ </span><span class="nb">sudo du</span> <span class="nt">-sh</span> <span class="k">*</span>
37M     blog-2021-04-26_01-31.tar.gz
3.4M    ghost_sitename_prod-2021-04-26_01-31.sql
</code></pre></div></div>

<h2 id="storage">Storage</h2>

<p><a href="https://cloud.google.com/free/docs/gcp-free-tier/#storage">https://cloud.google.com/free/docs/gcp-free-tier/#storage</a></p>

<p>project is still in testing phase, i’m going to attempt to spend $0 on my backup. With this goal in mind, i’ll setup gcs storage bucket, only keep 2 revisions, and delete objects older than 35 days, with the goal of backing up once a month.</p>

<p>The location matters, only few locations have free tier. Knowing this ahead of time, my GCE instance was also built in the same Region as where the storage will reside in an attempt to get FREE local transfer.</p>

<p>set some environement variables,</p>

<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">PROJECT_ID</span><span class="o">=</span>&lt;your project&gt;
<span class="nv">STORAGE_CLASS</span><span class="o">=</span>STANDARD
<span class="nv">BUCKET_LOCATION</span><span class="o">=</span>us-west1
</code></pre></div></div>

<p>create <code class="language-plaintext highlighter-rouge">lifecycle.json</code> and edit it to contain the following.</p>

<div class="language-json highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="p">{</span><span class="w">
</span><span class="nl">"lifecycle"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span><span class="w">
  </span><span class="nl">"rule"</span><span class="p">:</span><span class="w"> </span><span class="p">[</span><span class="w">
  </span><span class="p">{</span><span class="w">
    </span><span class="nl">"action"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span><span class="nl">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Delete"</span><span class="p">},</span><span class="w">
    </span><span class="nl">"condition"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span><span class="w">
      </span><span class="nl">"numNewerVersions"</span><span class="p">:</span><span class="w"> </span><span class="mi">2</span><span class="w">
    </span><span class="p">}</span><span class="w">
  </span><span class="p">},</span><span class="w">
  </span><span class="p">{</span><span class="w">
    </span><span class="nl">"action"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span><span class="nl">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Delete"</span><span class="p">},</span><span class="w">
    </span><span class="nl">"condition"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span><span class="w">
      </span><span class="nl">"age"</span><span class="p">:</span><span class="w"> </span><span class="mi">35</span><span class="p">,</span><span class="w">
      </span><span class="nl">"isLive"</span><span class="p">:</span><span class="w"> </span><span class="kc">false</span><span class="w">
    </span><span class="p">}</span><span class="w">
  </span><span class="p">}</span><span class="w">
</span><span class="p">]</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="p">}</span><span class="w">
</span></code></pre></div></div>

<p>create storage bucket and set the lifecycle policy and versioning.</p>

<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code>gsutil mb <span class="nt">-b</span> on <span class="nt">-p</span> <span class="nv">$PROJECT_ID</span> <span class="nt">-c</span> <span class="nv">$STORAGE_CLASS</span> <span class="nt">-l</span> <span class="nv">$BUCKET_LOCATION</span> <span class="nt">-b</span> on gs://<span class="nv">$PROJECT_ID</span><span class="nt">-ghost-backup</span> 
gsutil versioning <span class="nb">set </span>on gs://<span class="nv">$PROJECT_ID</span><span class="nt">-ghost-backup</span>
gsutil lifecycle <span class="nb">set </span>lifecycle.json gs://<span class="nv">$PROJECT_ID</span><span class="nt">-ghost-backup</span>
</code></pre></div></div>

<p>now let’s enable <a href="https://cloud.google.com/storage/docs/uniform-bucket-level-access">UBL</a>. UBL is this thing that allows you to enforce IAM roles/permissions (<code class="language-plaintext highlighter-rouge">roles/storage.objectViewer</code>) on a bucket (<code class="language-plaintext highlighter-rouge">gs://$PROJECT_ID-ghost-backup</code>) restricted only to a GCP service account (<code class="language-plaintext highlighter-rouge">serviceAccount:$NODE_SA_ID</code>).</p>

<p><code class="language-plaintext highlighter-rouge">NODE_SA_ID</code> is the service account I used to create the instance in <a href="/running-ghost-on-gcp">Running Ghost on GCP</a>, which will allow me to bypass having to deal with credentials management.</p>

<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code>gsutil iam ch serviceAccount:<span class="nv">$NODE_SA_ID</span>:roles/storage.objectViewer gs://<span class="nv">$PROJECT_ID</span><span class="nt">-ghost-backup</span>
gsutil iam ch serviceAccount:<span class="nv">$NODE_SA_ID</span>:roles/storage.objectCreator gs://<span class="nv">$PROJECT_ID</span><span class="nt">-ghost-backup</span>
</code></pre></div></div>

<p>if you set <code class="language-plaintext highlighter-rouge">objectViewer</code> and <code class="language-plaintext highlighter-rouge">objectCreator</code> at the ServiceAccount level, you can undo it by running the following</p>

<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code>gcloud projects remove-iam-policy-binding <span class="nv">$PROJECT_ID</span> <span class="nt">--member</span><span class="o">=</span>serviceAccount:<span class="k">${</span><span class="nv">NODE_SA_ID</span><span class="k">}</span> <span class="nt">--role</span><span class="o">=</span>roles/storage.objectCreator
gcloud projects remove-iam-policy-binding <span class="nv">$PROJECT_ID</span> <span class="nt">--member</span><span class="o">=</span>serviceAccount:<span class="k">${</span><span class="nv">NODE_SA_ID</span><span class="k">}</span> <span class="nt">--role</span><span class="o">=</span>roles/storage.objectViewer
</code></pre></div></div>

<h3 id="upload">Upload</h3>

<p>before let’s prepare, possible this is already in order</p>

<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">now</span><span class="o">=</span><span class="si">$(</span><span class="nb">date</span> +<span class="s1">'%Y-%m-%d_%H-%M'</span><span class="si">)</span>
<span class="nv">data_path</span><span class="o">=</span><span class="s2">"/tmp/data"</span>
<span class="nb">mkdir</span> <span class="nt">-p</span> <span class="s2">"</span><span class="nv">$data_path</span><span class="s2">"</span>
<span class="nb">sudo chown</span> <span class="nv">$USER</span> <span class="s2">"</span><span class="nv">$data_path</span><span class="s2">"</span>  
<span class="nb">sudo chmod </span>755 <span class="s2">"</span><span class="nv">$data_path</span><span class="s2">"</span> 
</code></pre></div></div>

<p>Now run the following</p>

<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code>docker run <span class="nt">--name</span> cloud-sdk <span class="nt">-it</span> <span class="se">\</span>
  <span class="nt">--restart</span><span class="o">=</span>always <span class="nt">-d</span> <span class="se">\</span>
  <span class="nt">-v</span> <span class="nv">$data_path</span>/backup:<span class="nv">$data_path</span>/backup <span class="se">\</span>
  <span class="nt">-e</span> <span class="nv">now</span><span class="o">=</span><span class="nv">$now</span> <span class="se">\</span>
  <span class="nt">-e</span> <span class="nv">data_path</span><span class="o">=</span><span class="nv">$data_path</span> <span class="se">\</span>
  <span class="nt">-e</span> <span class="nv">PROJECT_ID</span><span class="o">=</span><span class="nv">$PROJECT_ID</span> google/cloud-sdk
docker <span class="nb">exec</span> <span class="nt">-it</span> cloud-sdk /bash/bin/sh
</code></pre></div></div>

<p>once in the shell setup <code class="language-plaintext highlighter-rouge">gsutils</code> by running <code class="language-plaintext highlighter-rouge">gcloud init</code>, follow the prompts.</p>

<p>of course <code class="language-plaintext highlighter-rouge">gcloud auth list</code> can verify the caller.</p>

<p>while we’re at it let’s test our permissions</p>

<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c"># gsutil ls</span>
AccessDeniedException: 403 <span class="nv">$NODE_SA</span> does not have storage.buckets.list access to the Google Cloud project.
</code></pre></div></div>

<p>let’s upload the latest backup to the bucket</p>

<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code>gsutil <span class="nb">cp</span> <span class="nv">$data_path</span>/backup/blog-<span class="nv">$now</span>.tar.gz gs://<span class="nv">$PROJECT_ID</span><span class="nt">-ghost-backup</span>/blog.tar.gz
gsutil <span class="nb">cp</span> <span class="nv">$data_path</span>/backup/ghost-<span class="nv">$now</span>.sql gs://<span class="nv">$PROJECT_ID</span><span class="nt">-ghost-backup</span>/ghost.sql
</code></pre></div></div>

<p>now let’s verify the backup is there…</p>

<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code>gsutil <span class="nb">ls </span>gs://<span class="nv">$PROJECT_ID</span><span class="nt">-ghost-backup</span>/
</code></pre></div></div>

<h2 id="retrieve">Retrieve</h2>

<p>This seems redundant, but in an actual upgrade/move scenario, this will be ran from another instance.</p>

<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code>docker run <span class="nt">--name</span> cloud-sdk <span class="nt">-it</span> <span class="se">\</span>
  <span class="nt">--restart</span><span class="o">=</span>always <span class="nt">-d</span> <span class="se">\</span>
  <span class="nt">-v</span> <span class="nv">$data_path</span>/backup:<span class="nv">$data_path</span>/backup <span class="se">\</span>
  <span class="nt">-e</span> <span class="nv">now</span><span class="o">=</span><span class="nv">$now</span> <span class="se">\</span>
  <span class="nt">-e</span> <span class="nv">data_path</span><span class="o">=</span><span class="nv">$data_path</span> <span class="se">\</span>
  <span class="nt">-e</span> <span class="nv">PROJECT_ID</span><span class="o">=</span><span class="nv">$PROJECT_ID</span> google/cloud-sdk
docker <span class="nb">exec</span> <span class="nt">-it</span> cloud-sdk /bash/bin/sh
</code></pre></div></div>

<p>once again setup <code class="language-plaintext highlighter-rouge">gsutils</code> by running <code class="language-plaintext highlighter-rouge">gcloud init</code>, follow the prompts.</p>

<p>now let’s retrieve the backup</p>

<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code>gsutil <span class="nb">cp </span>gs://<span class="nv">$PROJECT_ID</span><span class="nt">-ghost-backup</span>/blog.tar.gz <span class="nv">$data_path</span>/backup/blog-<span class="nv">$now</span>.tar.gz
gsutil <span class="nb">cp </span>gs://<span class="nv">$PROJECT_ID</span><span class="nt">-ghost-backup</span>/ghost.sql <span class="nv">$data_path</span>/backup/ghost-<span class="nv">$now</span>.sql
</code></pre></div></div>

<p>now let’s cleanup</p>

<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code>docker <span class="nb">rm</span> <span class="nt">-f</span> cloud-sdk
</code></pre></div></div>

<h2 id="restore">Restore</h2>

<p>In <a href="/lets-encrypt-nginx-docker-ghost">Running ghost blog with nginx reverse proxy and let’s encrypt</a>, we created a mysql container and a ghost container to start the setup.</p>

<h3 id="db">DB</h3>

<p>Assuming I was to start from scratch, the first thing to do would be to create the mysql container, apply the fix for a small ghost mysql integration issue: <a href="https://github.com/mysqljs/mysql/issues/1507">https://github.com/mysqljs/mysql/issues/1507</a>, all of this is copiously documented in <a href="/lets-encrypt-nginx-docker-ghost">Running ghost blog with nginx reverse proxy and let’s encrypt</a> and restore the backup.</p>

<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code>docker <span class="nb">exec</span> <span class="nt">-i</span> mysql sh <span class="nt">-c</span> <span class="s1">'mysql -u root -p'</span><span class="s2">"</span><span class="k">${</span><span class="nv">mysql_local_pass</span><span class="k">}</span><span class="s2">"</span><span class="s1">' ghost'</span> &lt; <span class="nv">$data_path</span>/backup/ghost-<span class="nv">$now</span>.sql
</code></pre></div></div>

<h3 id="files">Files</h3>

<p>The second thing to do would be to restore the files before starting the ghost container.</p>

<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nb">mkdir</span> <span class="nt">-p</span> <span class="nv">$data_path</span>/restore
<span class="nb">tar</span> <span class="nt">-xvf</span> <span class="nv">$data_path</span>/backup/blog-<span class="nv">$now</span>.tar.gz <span class="nt">-C</span> <span class="nv">$data_path</span>/restore

<span class="nb">mv</span> <span class="nv">$data_path</span>/restore/blog/images <span class="nv">$data_path</span>/blog/
<span class="nb">mv</span> <span class="nv">$data_path</span>/restore/blog/themes/<span class="o">{</span>hue,Massively-master<span class="o">}</span> <span class="nv">$data_path</span>/blog/themes/
<span class="nb">mv</span> <span class="nv">$data_path</span>/restore/blog/data/redirects.json <span class="nv">$data_path</span>/blog/data/redirects.json
<span class="nb">mv</span> <span class="nv">$data_path</span>/restore/blog/settings/routes.yaml <span class="nv">$data_path</span>/blog/settings/routes.yaml
</code></pre></div></div>

<p>the remaining of <a href="/lets-encrypt-nginx-docker-ghost">Running ghost blog with nginx reverse proxy and let’s encrypt</a> should still be valid for creating certs and creating ways to auto-renew the certs</p>]]></content><author><name>Kenneth Massada</name></author><category term="ghost" /><category term="blog" /><category term="gcp" /><summary type="html"><![CDATA[I've been using the blog platform ghost for hosting tadbit.cc, and now twdspodcast.com. Migrating to Ghost 4, this post is explores Ghost maintenance and backups]]></summary></entry><entry><title type="html">Running ghost blog on GCP, using GCE</title><link href="https://blog.kmassada.com/running-ghost-on-gcp/" rel="alternate" type="text/html" title="Running ghost blog on GCP, using GCE" /><published>2021-04-25T04:00:00+00:00</published><updated>2021-04-25T04:00:00+00:00</updated><id>https://blog.kmassada.com/running-ghost-on-gcp</id><content type="html" xml:base="https://blog.kmassada.com/running-ghost-on-gcp/"><![CDATA[<p>I’ve been using the blog platform ghost for hosting tadbit.cc, and now twdspodcast.com. Using the Ghost 4 launch as an excuse to tidy up how I create the instances and run them on GCP using GCE instances. This guide will go through creating a GCE instance with minimal roles/scopes, trying to keeping instance options to the least priviledged and thinking about costs.</p>

<h2 id="pre-reqs">Pre-Reqs</h2>

<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>PROJECT_ID=&lt;your project&gt;
CUSTOM_ROLE=ghostGCEnode
</code></pre></div></div>

<h2 id="serivice-account-and-roles">Serivice Account and Roles</h2>

<p>let’s create the user</p>

<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nb">cat</span> <span class="o">&lt;&lt;</span> <span class="no">EOF</span><span class="sh"> &gt;&gt; </span><span class="nv">$HOME</span><span class="sh">/</span><span class="nv">$CUSTOM_ROLE</span><span class="sh">.yaml
title: </span><span class="nv">$CUSTOM_ROLE</span><span class="sh">
description: Role for least privileged GCE user running ghost 
stage: GA
includedPermissions:
- compute.networks.use
- compute.networks.useExternalIp
- compute.subnetworks.use
- compute.instances.osLogin
- compute.disks.use
</span><span class="no">EOF
</span></code></pre></div></div>

<p>create the role</p>

<div class="language-sh highlighter-rouge"><div class="highlight"><pre class="highlight"><code>gcloud iam roles create <span class="nv">$CUSTOM_ROLE</span> <span class="nt">--project</span><span class="o">=</span><span class="nv">$PROJECT_ID</span> <span class="se">\</span>
  <span class="nt">--file</span><span class="o">=</span><span class="nv">$HOME</span>/<span class="nv">$CUSTOM_ROLE</span>.yaml
</code></pre></div></div>

<p>create a service account</p>

<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nb">export </span><span class="nv">VM_NAME</span><span class="o">=</span>ghostv4
<span class="nb">export </span><span class="nv">NODE_SA</span><span class="o">=</span>gce-<span class="nv">$VM_NAME</span><span class="nt">-node-sa</span>
<span class="nb">export </span><span class="nv">ZONE</span><span class="o">=</span>us-west1-a

gcloud iam service-accounts create <span class="nv">$NODE_SA</span> <span class="nt">--display-name</span> <span class="s1">'GCE '</span><span class="s2">"</span><span class="k">${</span><span class="nv">VM_NAME</span><span class="k">}</span><span class="s2">"</span><span class="s1">' Node Service Account'</span> <span class="se">\</span>
<span class="o">&amp;&amp;</span> <span class="nb">sleep </span>10 <span class="o">&amp;&amp;</span> <span class="se">\</span>
<span class="nb">export </span><span class="nv">NODE_SA_ID</span><span class="o">=</span><span class="sb">`</span>gcloud iam service-accounts list <span class="nt">--format</span><span class="o">=</span><span class="s1">'value(email)'</span> <span class="nt">--filter</span><span class="o">=</span><span class="s1">'displayName:GCE '</span><span class="s2">"</span><span class="k">${</span><span class="nv">VM_NAME</span><span class="k">}</span><span class="s2">"</span><span class="s1">' Node Service Account'</span><span class="sb">`</span>
</code></pre></div></div>

<h3 id="bind-custom-role">bind custom role</h3>

<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code>gcloud projects add-iam-policy-binding <span class="nv">$PROJECT_ID</span> <span class="nt">--member</span><span class="o">=</span>serviceAccount:<span class="k">${</span><span class="nv">NODE_SA_ID</span><span class="k">}</span> <span class="nt">--role</span><span class="o">=</span>projects/<span class="nv">$PROJECT_ID</span>/roles/<span class="nv">$CUSTOM_ROLE</span>
</code></pre></div></div>

<h3 id="storage-roles-option-a">Storage roles (Option A)</h3>
<p>In the future will need logging and storing objects in GCE, might add <code class="language-plaintext highlighter-rouge">roles/logging.logWriter</code>, but for now only strictly need to write to buckets and read to bucket,  <code class="language-plaintext highlighter-rouge">roles/storage.objectCreator</code> and <code class="language-plaintext highlighter-rouge">roles/storage.objectViewer</code> will be sufficient…</p>

<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code>gcloud projects add-iam-policy-binding <span class="nv">$PROJECT_ID</span> <span class="nt">--member</span><span class="o">=</span>serviceAccount:<span class="k">${</span><span class="nv">NODE_SA_ID</span><span class="k">}</span> <span class="nt">--role</span><span class="o">=</span>roles/storage.objectCreator
gcloud projects add-iam-policy-binding <span class="nv">$PROJECT_ID</span> <span class="nt">--member</span><span class="o">=</span>serviceAccount:<span class="k">${</span><span class="nv">NODE_SA_ID</span><span class="k">}</span> <span class="nt">--role</span><span class="o">=</span>roles/storage.objectViewer
</code></pre></div></div>

<h3 id="storage-roles-option-b">Storage roles (Option B)</h3>

<p><a href="https://cloud.google.com/storage/docs/uniform-bucket-level-access">UBL</a>, could actually lock down access further..</p>

<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>gsutil iam ch serviceAccount:$NODE_SA_ID:roles/storage.objectViewer gs://_______BUCKET_____NAME_______
gsutil iam ch serviceAccount:$NODE_SA_ID:roles/storage.objectCreator gs://_______BUCKET_____NAME_______
</code></pre></div></div>

<h3 id="setup-firewall">Setup Firewall</h3>

<p>create fw rules, certbot uses tcp:80 for the ACME challenge, instead of closing port 80, we’ll use nginx to force redirect to https</p>

<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code>gcloud compute <span class="nt">--project</span><span class="o">=</span><span class="nv">$PROJECT_ID</span> firewall-rules create default-allow-<span class="nv">$VM_NAME</span> <span class="nt">--direction</span><span class="o">=</span>INGRESS <span class="nt">--priority</span><span class="o">=</span>1000 <span class="nt">--network</span><span class="o">=</span>default <span class="nt">--action</span><span class="o">=</span>ALLOW <span class="nt">--rules</span><span class="o">=</span>tcp:80,tcp:443 <span class="nt">--source-ranges</span><span class="o">=</span>0.0.0.0/0 <span class="nt">--target-tags</span><span class="o">=</span><span class="nv">$VM_NAME</span><span class="nt">-server</span>
</code></pre></div></div>

<h3 id="setup-image">Setup Image</h3>

<p>we run on COS</p>

<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">COS_STABLE</span><span class="o">=</span><span class="sb">`</span>gcloud compute images list <span class="nt">--format</span><span class="o">=</span><span class="s2">"value(NAME)"</span> <span class="nt">--filter</span><span class="o">=</span><span class="s2">"selfLink~cos-cloud AND family~stable"</span> <span class="nt">--limit</span><span class="o">=</span>1<span class="sb">`</span>
</code></pre></div></div>

<h3 id="launch-instance">Launch Instance</h3>

<p>create an instance with the Service account, later we can add <code class="language-plaintext highlighter-rouge">logging-write,storage-rw</code> to the scope for the scopes for future use of GCR and GCS</p>

<p>few cost cutting measures, they do have performance implications</p>
<ul>
  <li><a href="https://cloud.google.com/network-tiers">network tier standard</a></li>
  <li><a href="https://cloud.google.com/free/docs/gcp-free-tier#free-tier-usage-limits">zone us-west1-a</a> to qualify for free tier for GCS</li>
  <li><a href="https://cloud.google.com/compute/disks-image-pricing#disk">boot disk 10G</a></li>
  <li><a href="https://cloud.google.com/compute/docs/disks#disk-types">disk type pd-standard</a></li>
  <li>instance type e2-micro, estimated at $10/month, <a href="https://cloud.google.com/free/docs/gcp-free-tier#free-tier-usage-limits">f1-micro qualifies for free tier</a></li>
</ul>

<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code>gcloud compute instances create <span class="nv">$VM_NAME</span> <span class="se">\</span>
  <span class="nt">--image</span><span class="o">=</span><span class="nv">$COS_STABLE</span> <span class="se">\</span>
  <span class="nt">--image-project</span><span class="o">=</span>cos-cloud <span class="se">\</span>
  <span class="nt">--zone</span><span class="o">=</span><span class="nv">$ZONE</span> <span class="se">\</span>
  <span class="nt">--boot-disk-size</span><span class="o">=</span>10G <span class="se">\</span>
  <span class="nt">--boot-disk-type</span><span class="o">=</span>pd-standard <span class="se">\</span>
  <span class="nt">--machine-type</span><span class="o">=</span>e2-micro <span class="se">\</span>
  <span class="nt">--scopes</span><span class="o">=</span>compute-rw<span class="se">\</span>
  <span class="nt">--tags</span> <span class="nv">$VM_NAME</span><span class="nt">-server</span> <span class="se">\</span>
  <span class="nt">--network-tier</span> STANDARD <span class="se">\</span>
  <span class="nt">--network</span><span class="o">=</span>default <span class="se">\</span>
  <span class="nt">--service-account</span><span class="o">=</span><span class="nv">$NODE_SA_ID</span>
</code></pre></div></div>

<h3 id="launch-an-instance-with-an-external-ssd">Launch an Instance with an External SSD</h3>

<p>alternatively here’s a slightly increased performance version</p>
<ul>
  <li>with a 32GB ssd to preserve data</li>
  <li>e2-small instead of e2-micro for 2G of ram</li>
</ul>

<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code>gcloud compute instances create <span class="nv">$VM_NAME</span> <span class="se">\</span>
  <span class="nt">--image</span><span class="o">=</span><span class="nv">$COS_STABLE</span> <span class="se">\</span>
  <span class="nt">--image-project</span><span class="o">=</span>cos-cloud <span class="se">\</span>
  <span class="nt">--zone</span><span class="o">=</span><span class="nv">$ZONE</span> <span class="se">\</span>
  <span class="nt">--boot-disk-size</span><span class="o">=</span>10G <span class="se">\</span>
  <span class="nt">--boot-disk-type</span><span class="o">=</span>pd-standard <span class="se">\</span>
  <span class="nt">--machine-type</span><span class="o">=</span>e2-small <span class="se">\</span>
  <span class="nt">--scopes</span><span class="o">=</span>compute-rw<span class="se">\</span>
  <span class="nt">--tags</span> <span class="nv">$VM_NAME</span><span class="nt">-server</span> <span class="se">\</span>
  <span class="nt">--network-tier</span> STANDARD <span class="se">\</span>
  <span class="nt">--network</span><span class="o">=</span>default <span class="se">\</span>
  <span class="nt">--service-account</span><span class="o">=</span><span class="nv">$NODE_SA_ID</span> <span class="se">\</span>
 <span class="nt">--create-disk</span><span class="o">=</span><span class="nv">mode</span><span class="o">=</span>rw,size<span class="o">=</span>32,type<span class="o">=</span>projects/<span class="nv">$PROJECT_ID</span>/zones/<span class="nv">$ZONE</span>/diskTypes/pd-ssd,name<span class="o">=</span><span class="nv">$VM_NAME</span><span class="nt">-data</span>,device-name<span class="o">=</span><span class="nv">$VM_NAME</span><span class="nt">-data</span>
</code></pre></div></div>

<p>TLDR on folllowing the guide to <a href="https://cloud.google.com/container-optimized-os/docs/concepts/disks-and-filesystem">mount a disk on a container-optimized-os instance</a></p>

<p>You can find <code class="language-plaintext highlighter-rouge">DEVICE_NAME</code> by running <code class="language-plaintext highlighter-rouge">lsblk</code>. in our case the output looks like this</p>

<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code>~ <span class="nv">$ </span>lsblk
NAME      MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
sda         8:0    0   10G  0 disk 
|-sda1      8:1    0  5.9G  0 part /mnt/stateful_partition
|-sda2      8:2    0   16M  0 part 
|-sda3      8:3    0    2G  0 part 
| <span class="sb">`</span><span class="nt">-vroot</span> 253:0    0    2G  1 dm   /
|-sda4      8:4    0   16M  0 part 
|-sda5      8:5    0    2G  0 part 
|-sda6      8:6    0  512B  0 part 
|-sda7      8:7    0  512B  0 part 
|-sda8      8:8    0   16M  0 part /usr/share/oem
|-sda9      8:9    0  512B  0 part 
|-sda10     8:10   0  512B  0 part 
|-sda11     8:11   0    8M  0 part 
<span class="sb">`</span><span class="nt">-sda12</span>     8:12   0   32M  0 part 
sdb         8:16   0   32G  0 disk
</code></pre></div></div>

<p>we then use <code class="language-plaintext highlighter-rouge">sdb</code> in our script to format disk</p>

<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">DEVICE_NAME</span><span class="o">=</span>sdb
<span class="nv">MOUNT_DIR</span><span class="o">=</span><span class="nv">$HOSTNAME</span><span class="nt">-data</span>
<span class="nb">sudo </span>mkfs.ext4 <span class="nt">-m</span> 0 <span class="nt">-E</span> <span class="nv">lazy_itable_init</span><span class="o">=</span>0,lazy_journal_init<span class="o">=</span>0,discard /dev/<span class="nv">$DEVICE_NAME</span>
<span class="nb">sudo mkdir</span> <span class="nt">-p</span> /mnt/disks/<span class="nv">$MOUNT_DIR</span>
<span class="nb">sudo </span>mount <span class="nt">-o</span> discard,defaults /dev/<span class="nv">$DEVICE_NAME</span> /mnt/disks/<span class="nv">$MOUNT_DIR</span>
<span class="nb">sudo chmod </span>a+w /mnt/disks/<span class="nv">$MOUNT_DIR</span>
</code></pre></div></div>

<p>here’s the cloud config that will auto mount the disk, the equivalent of doing an <code class="language-plaintext highlighter-rouge">fstab</code> on debian</p>

<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nb">cat</span> <span class="o">&lt;&lt;</span> <span class="no">EOF</span><span class="sh"> &gt;&gt; </span><span class="nv">$HOME</span><span class="sh">/</span><span class="nv">$VM_NAME</span><span class="sh">-cloud-config.yaml
#cloud-config

bootcmd:
- fsck.ext4 -tvy /dev/sdb
- mkdir -p /mnt/disks/</span><span class="nv">$VM_NAME</span><span class="sh">-data
- mount -t ext4 -o discard,defaults /dev/sdb /mnt/disks/</span><span class="nv">$VM_NAME</span><span class="sh">-data
</span><span class="no">EOF
</span></code></pre></div></div>

<p>modify instance to have cloud-config</p>

<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code>gcloud compute instances add-metadata <span class="nv">$VM_NAME</span> <span class="se">\</span>
    <span class="nt">--metadata-from-file</span> user-data<span class="o">=</span><span class="nv">$VM_NAME</span><span class="nt">-cloud-config</span>.yaml
</code></pre></div></div>

<p>you can check by rebooting and observing <code class="language-plaintext highlighter-rouge">lsblk</code>, in our case it’s mounted</p>

<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code>~ <span class="nv">$ </span>lsblk
NAME      MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
sda         8:0    0   10G  0 disk 
|-sda1      8:1    0  5.9G  0 part /mnt/stateful_partition
|-sda2      8:2    0   16M  0 part 
|-sda3      8:3    0    2G  0 part 
| <span class="sb">`</span><span class="nt">-vroot</span> 253:0    0    2G  1 dm   /
|-sda4      8:4    0   16M  0 part 
|-sda5      8:5    0    2G  0 part 
|-sda6      8:6    0  512B  0 part 
|-sda7      8:7    0  512B  0 part 
|-sda8      8:8    0   16M  0 part /usr/share/oem
|-sda9      8:9    0  512B  0 part 
|-sda10     8:10   0  512B  0 part 
|-sda11     8:11   0    8M  0 part 
<span class="sb">`</span><span class="nt">-sda12</span>     8:12   0   32M  0 part 
sdb         8:16   0   32G  0 disk /mnt/disks/<span class="nv">$VM_NAME</span><span class="nt">-data</span>
</code></pre></div></div>

<h3 id="verify">Verify</h3>

<p>When you’ve accessed the instance, we can run <code class="language-plaintext highlighter-rouge">gcloud auth list</code> to confirm the caller, it is <code class="language-plaintext highlighter-rouge">$NODE_SA_ID</code></p>

<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>$ docker run --rm -ti google/cloud-sdk gcloud auth list
                    Credentialed Accounts
ACTIVE  ACCOUNT
*       $NODE_SA_ID

To set the active account, run:
    $ gcloud config set account `ACCOUNT`
</code></pre></div></div>]]></content><author><name>Kenneth Massada</name></author><category term="ghost" /><category term="blog" /><category term="gcp" /><summary type="html"><![CDATA[I've been using the blog platform ghost for hosting tadbit.cc, and now twdspodcast.com. Using the Ghost 4 launch as an excuse to tidy up how I create the instances and run them on GCP using GCE instances]]></summary></entry><entry><title type="html">A covid story and finding a passion</title><link href="https://blog.kmassada.com/finding-a-passion/" rel="alternate" type="text/html" title="A covid story and finding a passion" /><published>2021-01-08T04:00:00+00:00</published><updated>2021-01-08T04:00:00+00:00</updated><id>https://blog.kmassada.com/finding-a-passion</id><content type="html" xml:base="https://blog.kmassada.com/finding-a-passion/"><![CDATA[<p>I don’t think I was alone to have felt shell shock when I realized the world was about to take a huge time out due to this deadly thing that seemed to roam our streets, from country to country, with no end in sight. I also know I’ve not been alone in questioning everything: “What’s the point of this whole thing”?</p>

<p>There were a few instances prior to the beginning of the shutdown that seeded a question in my mind, but it started to take deeper roots as the spring bloomed. I’ve been mulling for a while. “How do I find the balance between what I do for a living and what can I do in my community?”</p>

<p>This is why I’ve always taken advantage of the opportunities granted by my Google badge: traveling to Africa to teach computer science, working locally with several organizations who focus on access to STEM for underrepresented groups, and volunteering in Google’s summer of computer science program.</p>

<p>That urge to want to contribute and uplift others has led to my current role of SME/Mentor. Yup! You heard it right! A sizable amount of what I do today is spent mentoring individuals but with the goal set on scaling the experience I’ve acquired through my journey in our ranks.</p>

<figure class="third">
	<img src="/assets/images/wisci-2017/IMG_0917.jpg" />
	<img src="/assets/images/wisci-2017/IMG_2818.jpg" />
	<img src="/assets/images/wisci-2017/IMG_20170804_171739.jpg" />
	<figcaption>Classes at MUST.</figcaption>
</figure>

<p>These questions, although born out of curiosity, also bear a painful truth. Like many, I struggled a lot through the pandemic. I say this understanding my struggles were not pegged to my survival. I’m thankful to have been in a field that has not experienced the effects of the pandemic in the way many others did.</p>

<p>Feeling useful to my community, feeling connected to my peers and my world as I’ve known it, fed my professional work. A typical day on campus was a true energy pill. I said hi to everyone, especially the support staff on the campuses, for whom I’m very thankful. I made a point to go find teammates wherever they were hiding. By the time I got to my desk I was locked in, ready to work with a lot to prove. I also got inspired by visiting other campuses and feeding off my team’s energy. I was on the verge of 21 separate offices visited before the pandemic.</p>

<p>The loss of most of my day to day interpersonal inspiration, coupled with some personal challenges made this period even more difficult. In this mirage I lost a dear friend to suicide. The impact of this left a hole in my soul. There are a few more episodes in my life that have left me stunned. As open and vulnerable I consider myself to be, and as outspoken as I am, I’m still unraveling each thread, I barely know what I’m feeling, let alone the words to describe it.</p>

<p>The radio silence of life and the many days spent alone, helped me understand what gets me through the day and what gets me through the quarter. I realized that for me to find fulfillment in my productivity, it must matter, it must make me feel like I’m contributing something useful, it must be interesting. But also it must make me feel full, glistening from within.</p>

<p>Work, as I’ve known it, made me feel super helpful. There’s another level of wings you grow when a kid you mentored is getting accepted to a prestigious computer science program, or one of your prodigies is now a sack of an engineer at a big tech firm. Things like this overjoy me.</p>

<p>Despite the enjoyment I was getting from these opportunities, I started to ask myself whether I was making the best use of my skills. What is it that only I could offer the world?</p>

<p>Sal Khan, creator of Khan Academy, and comedian Hasan Minhaj, host of the Patriot Act, discussed this question in an amazing 30 min interview (<a href="https://www.youtube.com/watch?v=mm0Y3ym-JUg">Hasan Minhaj on finding your gifts, being authentic, &amp; understanding yourself | Homeroom with Sal</a>.
They discussed striking similarities between challenges they faced coming from Southeast Asian cultures and my own experience coming from West Africa. Our parents wanted us to all be doctors, lawyers and engineers and nothing else. I jokingly affirmed to my father that I shall return home with 2 PhDs. Every now and then, I still wake up in a cold sweat stressed about getting started on the first. Imposter syndrome is real, especially when your sister is the fearless <a href="https://www.linkedin.com/in/karen-massada-md-693b97119">Karen Massada, MD (and surgeon)</a>.</p>

<p>That interview opened a new thread in my mind about exploring my gifts, while at the same time giving me comfort knowing it’s okay if what I pursue doesn’t end world hunger. It’s okay if my favorite secret talent is curating music (I have a playlist for everything). I found a lot of comfort in the idea of finding things that come easy to me and contributing that to the world.</p>

<p>I decided to launch a project that would have nothing to do with Tech. Or so I thought at the moment. At first it was gardening. I wasn’t particularly good at it. I even managed to kill a succulent.</p>

<p>Then on a long drive, heading out to camp on the oregon coast, my good friend had the idea of starting a podcast. Podcasting seemed to have all the aspects that I wanted. It was creative and didn’t have to do with tech. Right? It bridged the human connection I needed. It brought me closer to my community, while also bringing stories, entertainment, and lessons to the world.</p>

<iframe src="https://open.spotify.com/embed-podcast/show/12ZzuLAPlcYSEwayDsYn03" width="100%" height="232" frameborder="0" allowtransparency="true" allow="encrypted-media"></iframe>

<p>I launched the project under the disguise of fun conversations with friends that span all types of topics: religion, spirituality, growth, geopolitics, economics, love. Remember my hidden passion for curating music? I found a way to include it by reaching out to up and coming artists to let me feature their music on my show.  I named this project the “Things We Don’t Say” podcast, TWDS <a href="https://twdspodcast.com">twdspodcast.com</a>.</p>

<p>I’ve debated sharing this in my Professional Networks since a lot of the original conversations revolved around my personal interests and not my techie/professional side. When I thought about mentioning TWDS podcast in these spaces, I realized that I felt a bit of shame around the project. Because of the erroneous thinking I’ve harvested growing up, I convinced myself certain things hold a different value than others. Like I had been taught, I believed that my value and contribution to the world didn’t matter unless it was through being a doctor, engineer, lawyer or somehow saved the world.</p>

<p>However, there’s a tremendous amount of rigor that has gone into bringing this to life, and I know a lot of my peers/mentors will be proud of the journey. Most importantly I’ve learned to be proud of my journey. I hope my future self can read this and find comfort in it, because I know I’ll be getting better at this for years to come.</p>

<p>Before this post gets too long, I must talk a bit about the “black story”. Relax. I’m not about to make you uncomfortable talking about race. Okay maybe, just a little.</p>

<p>As a people, African American, African, Afro-Carribean, Afro indigenous, it is paramount that we tell our stories today because we have a lot of missing pieces throughout history. In an episode of TWDS, my friend Raymond made a pertinent remark: “We are making history today”. Although the assiduous task of recovering our history is important it’s own right, it’s also the only way to be sure that history doesn’t repeat itself. We’ve been bad at recording our lived experiences and even worse about preserving it. This is why you’ll hear a lot of the stories on my platform come from voices you aren’t used to hearing from.</p>

<p>Okay, wait a minute, you thought I was finished? (If there’s a <a href="https://youtu.be/otwGb-S0txk?t=95">melody</a> that comes to your head after I’ve made that millennial dad joke, kudos to you! “<a href="https://www.youtube.com/watch?v=A8q8PXoJwVk">Cool</a>” in John Oliver’s voice).</p>

<p>The hilarious part about this project is, it quickly turned technical.</p>

<p>I’m currently working on being my own podcast platform host because I know a thing or two about distributed systems in the cloud. In the process I built a distributed site on GCP, using Let’s Encrypt for auto renewing my certs. I debated Traefik vs Nginx proxy for routing, used Facebook developer tools to detect actions and act on them, and built a website analytics pipeline to understand my listenership.</p>

<p>I’ve also picked up some unusual skills, including some Adobe Creative Suite skills because that’s what the cool kids do. I now understand why so many of my dear co-workers are audiophiles. Sound engineering is the most geeky thing I could do. I had to learn to De-ess, de-verb, gain-translate to UFS, compress, normalize, before I could deliver a clip.</p>

<p>The business aspect is also a fascinating component. I now have more understanding about branding, impressions, click through rate, downloads vs listenership, understanding and growing your audience, and even a tad bit of marketing. I think it’ll be noteworthy to someday talk about all the hats I’ve held at this point in time to bring to you my first season.</p>

<blockquote class="instagram-media" data-instgrm-captioned="" data-instgrm-permalink="https://www.instagram.com/p/CITnubGhQWu/?utm_source=ig_embed&amp;utm_campaign=loading" data-instgrm-version="13" style=" background:#FFF; border:0; border-radius:3px; box-shadow:0 0 1px 0 rgba(0,0,0,0.5),0 1px 10px 0 rgba(0,0,0,0.15); margin: 1px; max-width:540px; min-width:326px; padding:0; width:99.375%; width:-webkit-calc(100% - 2px); width:calc(100% - 2px);"><div style="padding:16px;"> <a href="https://www.instagram.com/p/CITnubGhQWu/?utm_source=ig_embed&amp;utm_campaign=loading" style=" background:#FFFFFF; line-height:0; padding:0 0; text-align:center; text-decoration:none; width:100%;" target="_blank"> <div style=" display: flex; flex-direction: row; align-items: center;"> <div style="background-color: #F4F4F4; border-radius: 50%; flex-grow: 0; height: 40px; margin-right: 14px; width: 40px;"></div> <div style="display: flex; flex-direction: column; flex-grow: 1; justify-content: center;"> <div style=" background-color: #F4F4F4; border-radius: 4px; flex-grow: 0; height: 14px; margin-bottom: 6px; width: 100px;"></div> <div style=" background-color: #F4F4F4; border-radius: 4px; flex-grow: 0; height: 14px; width: 60px;"></div></div></div><div style="padding: 19% 0;"></div> <div style="display:block; height:50px; margin:0 auto 12px; width:50px;"><svg width="50px" height="50px" viewBox="0 0 60 60" version="1.1" xmlns="https://www.w3.org/2000/svg" xmlns:xlink="https://www.w3.org/1999/xlink"><g stroke="none" stroke-width="1" fill="none" fill-rule="evenodd"><g transform="translate(-511.000000, -20.000000)" fill="#000000"><g><path d="M556.869,30.41 C554.814,30.41 553.148,32.076 553.148,34.131 C553.148,36.186 554.814,37.852 556.869,37.852 C558.924,37.852 560.59,36.186 560.59,34.131 C560.59,32.076 558.924,30.41 556.869,30.41 M541,60.657 C535.114,60.657 530.342,55.887 530.342,50 C530.342,44.114 535.114,39.342 541,39.342 C546.887,39.342 551.658,44.114 551.658,50 C551.658,55.887 546.887,60.657 541,60.657 M541,33.886 C532.1,33.886 524.886,41.1 524.886,50 C524.886,58.899 532.1,66.113 541,66.113 C549.9,66.113 557.115,58.899 557.115,50 C557.115,41.1 549.9,33.886 541,33.886 M565.378,62.101 C565.244,65.022 564.756,66.606 564.346,67.663 C563.803,69.06 563.154,70.057 562.106,71.106 C561.058,72.155 560.06,72.803 558.662,73.347 C557.607,73.757 556.021,74.244 553.102,74.378 C549.944,74.521 548.997,74.552 541,74.552 C533.003,74.552 532.056,74.521 528.898,74.378 C525.979,74.244 524.393,73.757 523.338,73.347 C521.94,72.803 520.942,72.155 519.894,71.106 C518.846,70.057 518.197,69.06 517.654,67.663 C517.244,66.606 516.755,65.022 516.623,62.101 C516.479,58.943 516.448,57.996 516.448,50 C516.448,42.003 516.479,41.056 516.623,37.899 C516.755,34.978 517.244,33.391 517.654,32.338 C518.197,30.938 518.846,29.942 519.894,28.894 C520.942,27.846 521.94,27.196 523.338,26.654 C524.393,26.244 525.979,25.756 528.898,25.623 C532.057,25.479 533.004,25.448 541,25.448 C548.997,25.448 549.943,25.479 553.102,25.623 C556.021,25.756 557.607,26.244 558.662,26.654 C560.06,27.196 561.058,27.846 562.106,28.894 C563.154,29.942 563.803,30.938 564.346,32.338 C564.756,33.391 565.244,34.978 565.378,37.899 C565.522,41.056 565.552,42.003 565.552,50 C565.552,57.996 565.522,58.943 565.378,62.101 M570.82,37.631 C570.674,34.438 570.167,32.258 569.425,30.349 C568.659,28.377 567.633,26.702 565.965,25.035 C564.297,23.368 562.623,22.342 560.652,21.575 C558.743,20.834 556.562,20.326 553.369,20.18 C550.169,20.033 549.148,20 541,20 C532.853,20 531.831,20.033 528.631,20.18 C525.438,20.326 523.257,20.834 521.349,21.575 C519.376,22.342 517.703,23.368 516.035,25.035 C514.368,26.702 513.342,28.377 512.574,30.349 C511.834,32.258 511.326,34.438 511.181,37.631 C511.035,40.831 511,41.851 511,50 C511,58.147 511.035,59.17 511.181,62.369 C511.326,65.562 511.834,67.743 512.574,69.651 C513.342,71.625 514.368,73.296 516.035,74.965 C517.703,76.634 519.376,77.658 521.349,78.425 C523.257,79.167 525.438,79.673 528.631,79.82 C531.831,79.965 532.853,80.001 541,80.001 C549.148,80.001 550.169,79.965 553.369,79.82 C556.562,79.673 558.743,79.167 560.652,78.425 C562.623,77.658 564.297,76.634 565.965,74.965 C567.633,73.296 568.659,71.625 569.425,69.651 C570.167,67.743 570.674,65.562 570.82,62.369 C570.966,59.17 571,58.147 571,50 C571,41.851 570.966,40.831 570.82,37.631"></path></g></g></g></svg></div><div style="padding-top: 8px;"> <div style=" color:#3897f0; font-family:Arial,sans-serif; font-size:14px; font-style:normal; font-weight:550; line-height:18px;"> View this post on Instagram</div></div><div style="padding: 12.5% 0;"></div> <div style="display: flex; flex-direction: row; margin-bottom: 14px; align-items: center;"><div> <div style="background-color: #F4F4F4; border-radius: 50%; height: 12.5px; width: 12.5px; transform: translateX(0px) translateY(7px);"></div> <div style="background-color: #F4F4F4; height: 12.5px; transform: rotate(-45deg) translateX(3px) translateY(1px); width: 12.5px; flex-grow: 0; margin-right: 14px; margin-left: 2px;"></div> <div style="background-color: #F4F4F4; border-radius: 50%; height: 12.5px; width: 12.5px; transform: translateX(9px) translateY(-18px);"></div></div><div style="margin-left: 8px;"> <div style=" background-color: #F4F4F4; border-radius: 50%; flex-grow: 0; height: 20px; width: 20px;"></div> <div style=" width: 0; height: 0; border-top: 2px solid transparent; border-left: 6px solid #f4f4f4; border-bottom: 2px solid transparent; transform: translateX(16px) translateY(-4px) rotate(30deg)"></div></div><div style="margin-left: auto;"> <div style=" width: 0px; border-top: 8px solid #F4F4F4; border-right: 8px solid transparent; transform: translateY(16px);"></div> <div style=" background-color: #F4F4F4; flex-grow: 0; height: 12px; width: 16px; transform: translateY(-4px);"></div> <div style=" width: 0; height: 0; border-top: 8px solid #F4F4F4; border-left: 8px solid transparent; transform: translateY(-4px) translateX(8px);"></div></div></div> <div style="display: flex; flex-direction: column; flex-grow: 1; justify-content: center; margin-bottom: 24px;"> <div style=" background-color: #F4F4F4; border-radius: 4px; flex-grow: 0; height: 14px; margin-bottom: 6px; width: 224px;"></div> <div style=" background-color: #F4F4F4; border-radius: 4px; flex-grow: 0; height: 14px; width: 144px;"></div></div></a><p style=" color:#c9c8cd; font-family:Arial,sans-serif; font-size:14px; line-height:17px; margin-bottom:0; margin-top:8px; overflow:hidden; padding:8px 0 7px; text-align:center; text-overflow:ellipsis; white-space:nowrap;"><a href="https://www.instagram.com/p/CITnubGhQWu/?utm_source=ig_embed&amp;utm_campaign=loading" style=" color:#c9c8cd; font-family:Arial,sans-serif; font-size:14px; font-style:normal; font-weight:normal; line-height:17px; text-decoration:none;" target="_blank">A post shared by TWDS podcast (@twdspodcast)</a></p></div></blockquote>
<script async="" src="//www.instagram.com/embed.js"></script>

<p>You know that inertia from throwing an object you expect to go far, but just watching it fly a few feet at best! That’s what I feel. However, I’m proud that I lunged, put my hips into it and let loose. I’m really excited because it’s the first step I’ve ever taken out of my trade, so may we toast to many more. Maybe after this season no other episode will see the day, but I will always be able to look back at the courage it took to step out of my comfort zone.</p>

<p>Hope you’ve enjoyed this read.</p>]]></content><author><name>Kenneth Massada</name></author><category term="mystory" /><category term="blog" /><summary type="html"><![CDATA[This is the story of wrestling with the meaning of all through the pandemic, finding a passion, feeling shame to share the passion, finding pride in what you do, even if it doesn't save the world, being proud of yourself, and a rare occasion to tell my story.]]></summary></entry><entry><title type="html">Running ghost blog with nginx reverse proxy and let’s encrypt</title><link href="https://blog.kmassada.com/lets-encrypt-nginx-docker-ghost/" rel="alternate" type="text/html" title="Running ghost blog with nginx reverse proxy and let’s encrypt" /><published>2020-10-20T04:00:00+00:00</published><updated>2020-10-20T04:00:00+00:00</updated><id>https://blog.kmassada.com/lets-encrypt-nginx-docker-ghost</id><content type="html" xml:base="https://blog.kmassada.com/lets-encrypt-nginx-docker-ghost/"><![CDATA[<p>I’ve been using the blog platform ghost for hosting <a href="https://tadbit.cc">tadbit</a>, and now <a href="https://twdspodcast.com">twdspodcast</a>, this is a quick brain dump on how I host the blog. It’s simply nginx reverse proxy, ghost platform backed by mysql, and now the reverse proxy gets it’s cert from let’s encrypt. This is let’s get this to work fast and re-iterate later.</p>

<p>I create a gcp instance only running container optimized OS, allow http/https with a public IP. This public IP becomes A record in my domain records.</p>

<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>@      A      1h       ##.##.##.###
</code></pre></div></div>

<p>I ssh into the instance in the UI</p>

<h2 id="environment">Environment</h2>

<p>First I create a <code class="language-plaintext highlighter-rouge">.env</code> file to keep all the variables I need for my setup.</p>

<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>domain=example.org
email="admin@$domain" 
mysql_local_pass=STUFFMAN
data_path="/tmp/data"
domain_short=$(echo $domain | tr '.-' '_')
</code></pre></div></div>

<p><em>quick note:</em> /tmp/ is not a good path for saving data… in <a href="/running-ghost-on-gcp">Running Ghost on GCP</a> I create a disk that I mount to /mnt/disks/$VM_NAME-data, I omit this in this setup.</p>

<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>source .env
</code></pre></div></div>

<p>Now that this is sourced, let’s move on to running the blog platform itself.</p>

<p>NOTE: mail provider here is mailgun.com, create a domain and get username and password. <a href="https://ghost.org/docs/config/#mail">More info</a></p>

<h2 id="house-keeping">House keeping</h2>

<p>We create a network, this network is super helpful so we don’t manage ips and routes for container to container</p>

<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code>docker network create znet
<span class="nb">mkdir</span> <span class="nt">-p</span> <span class="s2">"</span><span class="nv">$data_path</span><span class="s2">"</span>
<span class="nb">sudo chown</span> <span class="nv">$USER</span> <span class="s2">"</span><span class="nv">$data_path</span><span class="s2">"</span>  
<span class="nb">sudo chmod </span>755 <span class="s2">"</span><span class="nv">$data_path</span><span class="s2">"</span>  
<span class="nb">mkdir</span> <span class="nt">-p</span> <span class="s2">"</span><span class="nv">$data_path</span><span class="s2">/</span><span class="k">${</span><span class="nv">domain_short</span><span class="k">}</span><span class="s2">_ghost"</span>
<span class="nb">mkdir</span> <span class="nt">-p</span> <span class="s2">"</span><span class="nv">$data_path</span><span class="s2">/mysql"</span>
</code></pre></div></div>

<h2 id="ghost">Ghost</h2>

<p>Run an instance of mysql first.</p>

<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code>docker run <span class="nt">--name</span><span class="o">=</span>mysql <span class="nt">--restart</span><span class="o">=</span>always <span class="nt">-d</span> <span class="nt">-p</span> 3306:3306 <span class="se">\</span>
 <span class="nt">--net</span><span class="o">=</span>znet <span class="se">\</span>
 <span class="nt">-v</span> <span class="nv">$data_path</span>/mysql:/var/lib/mysql <span class="se">\</span>
<span class="nt">-e</span> <span class="nv">MYSQL_ROOT_PASSWORD</span><span class="o">=</span><span class="nv">$mysql_local_pass</span> mysql
</code></pre></div></div>

<p>use those credentials to then run the blog</p>

<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code>docker run <span class="nt">--name</span><span class="o">=</span><span class="k">${</span><span class="nv">domain_short</span><span class="k">}</span>_ghost <span class="nt">--restart</span><span class="o">=</span>always <span class="nt">-d</span> <span class="se">\</span>
<span class="nt">-v</span> <span class="nv">$data_path</span>/<span class="k">${</span><span class="nv">domain_short</span><span class="k">}</span>_ghost:/var/lib/ghost/content <span class="se">\</span>
<span class="nt">-p</span> 3001:2368 <span class="se">\</span>
 <span class="nt">--net</span><span class="o">=</span>znet <span class="se">\</span>
<span class="nt">-e</span> <span class="nv">url</span><span class="o">=</span>http://<span class="nv">$domain</span> <span class="se">\</span>
<span class="nt">-e</span> <span class="nv">database__client</span><span class="o">=</span>mysql <span class="se">\</span>
<span class="nt">-e</span> <span class="nv">database__connection__host</span><span class="o">=</span>mysql <span class="se">\</span>
<span class="nt">-e</span> <span class="nv">database__connection__user</span><span class="o">=</span>root <span class="se">\</span>
<span class="nt">-e</span> <span class="nv">database__connection__password</span><span class="o">=</span><span class="nv">$mysql_local_pass</span> <span class="se">\</span>
<span class="nt">-e</span> <span class="nv">database__connection__database</span><span class="o">=</span><span class="k">${</span><span class="nv">domain_short</span><span class="k">}</span>_ghost <span class="se">\</span>
<span class="nt">-e</span> <span class="nv">mail__transport</span><span class="o">=</span><span class="s2">"SMTP"</span> <span class="se">\</span>
<span class="nt">-e</span> <span class="nv">mail__from</span><span class="o">=</span><span class="s2">"</span><span class="nv">$mail_name</span><span class="s2"> &lt;</span><span class="nv">$mail_username</span><span class="s2">&gt;"</span> <span class="se">\</span>
<span class="nt">-e</span> <span class="nv">mail__options__service</span><span class="o">=</span><span class="s2">"SMTP"</span> <span class="se">\</span>
<span class="nt">-e</span> <span class="nv">mail__options__host</span><span class="o">=</span><span class="s2">"</span><span class="nv">$mail_provider</span><span class="s2">"</span> <span class="se">\</span>
<span class="nt">-e</span> <span class="nv">mail__options__port</span><span class="o">=</span><span class="s2">"587"</span> <span class="se">\</span>
<span class="nt">-e</span> <span class="nv">mail__options__auth__user</span><span class="o">=</span><span class="s2">"</span><span class="nv">$mail_username</span><span class="s2">"</span> <span class="se">\</span>
<span class="nt">-e</span> <span class="nv">mail__options__auth__pass</span><span class="o">=</span><span class="s2">"</span><span class="nv">$mail_password</span><span class="s2">"</span> <span class="se">\</span>
ghost
</code></pre></div></div>

<h3 id="quirk-of-mysql">quirk of mysql</h3>

<p>I did run into a small issue with how ghost authenticates against mysql, it’s well documented: <a href="https://github.com/mysqljs/mysql/issues/1507">https://github.com/mysqljs/mysql/issues/1507</a></p>

<p>this was the quick fix</p>

<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nb">cat</span> <span class="o">&lt;&lt;</span> <span class="no">EOF</span><span class="sh"> &gt;&gt; </span><span class="nv">$data_path</span><span class="sh">/1507-fix.sql 
ALTER USER 'root'@'%' IDENTIFIED WITH mysql_native_password BY '</span><span class="nv">$mysql_local_pass</span><span class="sh">';
ALTER USER 'root'@'localhost' IDENTIFIED WITH mysql_native_password BY '</span><span class="nv">$mysql_local_pass</span><span class="sh">';
SELECT plugin FROM mysql.user WHERE User = 'root';
commit;
</span><span class="no">EOF
</span></code></pre></div></div>

<p>run quick fix file</p>

<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code>docker <span class="nb">exec</span> <span class="nt">-i</span> mysql sh <span class="nt">-c</span> <span class="s1">'mysql -u root -p'</span><span class="s2">"</span><span class="k">${</span><span class="nv">mysql_local_pass</span><span class="k">}</span><span class="s2">"</span><span class="s1">''</span> &lt; <span class="nv">$data_path</span>/1507-fix.sql
</code></pre></div></div>

<p>in my case after the quick fix, restart ghost <code class="language-plaintext highlighter-rouge">docker restart ${domain_short}_ghost</code></p>

<p>there’s a new problem that occurs in the newer versions of ghost. It thinks we have performed a migration, here’s a quick fix for it.</p>

<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nb">cat</span> <span class="o">&lt;&lt;</span> <span class="no">EOF</span><span class="sh"> &gt;&gt; </span><span class="nv">$data_path</span><span class="sh">/migration-lock-fix.sql 
USE </span><span class="k">${</span><span class="nv">domain_short</span><span class="k">}</span><span class="sh">_ghost;
UPDATE migrations_lock set locked=0 where lock_key='km01';
commit;
</span><span class="no">EOF
</span></code></pre></div></div>
<p>run quick fix file</p>

<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code>docker <span class="nb">exec</span> <span class="nt">-i</span> mysql sh <span class="nt">-c</span> <span class="s1">'mysql -u root -p'</span><span class="s2">"</span><span class="k">${</span><span class="nv">mysql_local_pass</span><span class="k">}</span><span class="s2">"</span><span class="s1">''</span> &lt; <span class="nv">$data_path</span>/migration-lock-fix.sql
</code></pre></div></div>

<p>in my case after the quick fix, restart ghost <code class="language-plaintext highlighter-rouge">docker restart ${domain_short}_ghost</code></p>

<h2 id="nginx--certbot">Nginx + certbot</h2>

<p>First we download the recommended ssl configs for nginx provided by certbot and ssl params, then we make a make belief cert that we later delete, we configure nginx and start it with make belief cert. Now when that’s complete we use certbot to gain staging cert, when successful we get production certs.</p>

<h3 id="download-ssl-conf-and-ssl-dhparamspem">Download SSL Conf and ssl-dhparams.pem</h3>

<p>Download required files from github.</p>

<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nb">mkdir</span> <span class="nt">-p</span> <span class="s2">"</span><span class="nv">$data_path</span><span class="s2">/ssl"</span>
curl <span class="nt">-s</span> https://raw.githubusercontent.com/certbot/certbot/master/certbot-nginx/certbot_nginx/_internal/tls_configs/options-ssl-nginx.conf <span class="o">&gt;</span> <span class="s2">"</span><span class="nv">$data_path</span><span class="s2">/ssl/options-ssl-nginx.conf"</span>
curl <span class="nt">-s</span> https://raw.githubusercontent.com/certbot/certbot/master/certbot/certbot/ssl-dhparams.pem <span class="o">&gt;</span> <span class="s2">"</span><span class="nv">$data_path</span><span class="s2">/ssl/ssl-dhparams.pem"</span>
</code></pre></div></div>

<h3 id="make-belief-certificate">make belief certificate</h3>

<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nb">mkdir</span> <span class="nt">-p</span> <span class="s2">"</span><span class="nv">$data_path</span><span class="s2">/ssl/live/</span><span class="nv">$domain</span><span class="s2">"</span>
</code></pre></div></div>

<p>use the nginx pod to run openssl for obtaining make belief cert</p>

<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code>docker run  <span class="nt">--rm</span> <span class="nt">-it</span> <span class="se">\</span>
  <span class="nt">-v</span> <span class="nv">$data_path</span>/ssl:/etc/letsencrypt <span class="nt">-v</span> <span class="nv">$data_path</span>/www:/var/www/certbot <span class="se">\</span>
   nginx sh <span class="nt">-c</span> <span class="s2">"mkdir -p /etc/letsencrypt/live/</span><span class="nv">$domain</span><span class="s2"> &amp;&amp; openssl req -x509 -nodes -newkey rsa:2048 -days 1</span><span class="se">\</span><span class="s2">
    -keyout '/etc/letsencrypt/live/</span><span class="nv">$domain</span><span class="s2">/privkey.pem' </span><span class="se">\</span><span class="s2">
    -out '/etc/letsencrypt/live/</span><span class="nv">$domain</span><span class="s2">/fullchain.pem' </span><span class="se">\</span><span class="s2">
    -subj '/CN=localhost'"</span>
</code></pre></div></div>

<h3 id="configure-nginx-reverse-proxy-for-ghost-w-ssl">configure nginx reverse proxy for ghost w/ ssl</h3>

<p>Setup the file</p>

<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nb">mkdir</span> <span class="nt">-p</span> <span class="nv">$data_path</span>/nginx/
vi <span class="nv">$data_path</span>/nginx/<span class="nv">$domain_short</span>.conf
</code></pre></div></div>

<p>Few things to know</p>

<ul>
  <li><code class="language-plaintext highlighter-rouge">http://${domain_short}_ghost:2368</code> is the path direct to the ghost container, because they share <code class="language-plaintext highlighter-rouge">znet</code> network</li>
  <li>many settings look like they are missing but they come from <code class="language-plaintext highlighter-rouge">options-ssl-nginx.conf</code>. as we downloaded earlier.</li>
  <li>in the config below please make sure to replace <code class="language-plaintext highlighter-rouge">example.org</code> by your domain and <code class="language-plaintext highlighter-rouge">example_org</code> by your domain short</li>
</ul>

<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code>server <span class="o">{</span>

  listen 443 ssl<span class="p">;</span>
  server_name example.org<span class="p">;</span>

  <span class="c"># this include is the recommended ssl settings by let's encrypt</span>
  include /etc/letsencrypt/options-ssl-nginx.conf<span class="p">;</span>

  add_header Strict-Transport-Security    <span class="s2">"max-age=31536000; includeSubDomain"</span> always<span class="p">;</span>
  add_header X-Frame-Options              SAMEORIGIN<span class="p">;</span>
  add_header X-Content-Type-Options       nosniff<span class="p">;</span>
  add_header X-XSS-Protection             <span class="s2">"1; mode=block"</span><span class="p">;</span>


  <span class="c"># this is the dhparam we downloaded from the onset</span>
  ssl_dhparam                 /etc/letsencrypt/ssl-dhparams.pem<span class="p">;</span>
  ssl_certificate             /etc/letsencrypt/live/example.org/fullchain.pem<span class="p">;</span>
  ssl_certificate_key         /etc/letsencrypt/live/example.org/privkey.pem<span class="p">;</span>

  <span class="c"># sometimes exporting this is useful too</span>
  access_log            /var/log/nginx/example.org.access.log<span class="p">;</span>

  location / <span class="o">{</span>

    proxy_set_header    X-Real-IP           <span class="nv">$remote_addr</span><span class="p">;</span>      
    proxy_set_header    X-Forwarded-For     <span class="nv">$proxy_add_x_forwarded_for</span><span class="p">;</span>
    proxy_set_header    X-Forwarded-Proto   <span class="nv">$scheme</span><span class="p">;</span>
    proxy_set_header    Host                <span class="nv">$host</span><span class="p">;</span>
    proxy_set_header    X-Forwarded-Host    <span class="nv">$host</span><span class="p">;</span>
    proxy_set_header    X-Forwarded-Port    <span class="nv">$server_port</span><span class="p">;</span>

    <span class="c"># Fix the “It appears that your reverse proxy set up is broken" error.</span>
    proxy_pass          http://example_org_ghost:2368<span class="p">;</span>
    proxy_read_timeout  90<span class="p">;</span>

    proxy_redirect      http://example_org_ghost:2368 https://example.org<span class="p">;</span>
  <span class="o">}</span>
<span class="o">}</span>
</code></pre></div></div>

<h3 id="start-nginx">start nginx</h3>

<p>this stage is important because here we are testing the vailidity of our nginx reverse proxy settings. So we test that the certs are placed in the correct place and that requests to reverse proxy reaches the app.</p>

<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code>docker run  <span class="nt">--name</span><span class="o">=</span>nginx <span class="nt">--restart</span><span class="o">=</span>always <span class="nt">-d</span> <span class="se">\</span>
  <span class="nt">-v</span> <span class="nv">$data_path</span>/ssl:/etc/letsencrypt <span class="nt">-v</span> <span class="nv">$data_path</span>/www:/var/www/certbot <span class="se">\</span>
  <span class="nt">-v</span> <span class="nv">$data_path</span>/nginx:/etc/nginx/conf.d <span class="se">\</span>
  <span class="nt">-p</span> 80:80 <span class="nt">-p</span> 443:443 <span class="se">\</span>
   <span class="nt">--net</span><span class="o">=</span>znet <span class="se">\</span>
   nginx 
</code></pre></div></div>

<p>now we delete the dummies with confidence</p>

<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code>docker <span class="nb">exec</span> <span class="nt">-it</span> nginx sh <span class="nt">-c</span> <span class="s2">"rm -Rvf /etc/letsencrypt/live/</span><span class="nv">$domain</span><span class="s2"> &amp;&amp; </span><span class="se">\</span><span class="s2">
  rm -Rvf /etc/letsencrypt/archive/</span><span class="nv">$domain</span><span class="s2"> &amp;&amp; </span><span class="se">\</span><span class="s2">
  rm -Rvf /etc/letsencrypt/renewal/</span><span class="nv">$domain</span><span class="s2">.conf"</span>
</code></pre></div></div>

<h3 id="configure-nginx-pre-stage">configure nginx pre-stage</h3>

<p>Setup the file</p>

<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nb">mkdir</span> <span class="nt">-p</span> <span class="nv">$data_path</span>/nginx/
vi <span class="nv">$data_path</span>/nginx/validate.conf
</code></pre></div></div>

<p>we configure nginx to allow for a callback to <code class="language-plaintext highlighter-rouge">/.well-known/acme-challenge/</code>. this will allow the staging command to verify ownership of the domain</p>

<div class="language-conf highlighter-rouge"><div class="highlight"><pre class="highlight"><code>
<span class="n">server</span> {
  <span class="n">listen</span>        <span class="m">80</span>;
  <span class="n">server_name</span>   <span class="err">_</span>;

  <span class="n">location</span> / {
    <span class="n">return</span> <span class="m">301</span> <span class="n">https</span>://$<span class="n">host</span>$<span class="n">request_uri</span>;
  }

  <span class="n">location</span> /.<span class="n">well</span>-<span class="n">known</span>/<span class="n">acme</span>-<span class="n">challenge</span>/ {
    <span class="n">root</span> /<span class="n">var</span>/<span class="n">www</span>/<span class="n">certbot</span>;
    <span class="n">allow</span> <span class="n">all</span>;
    <span class="n">try_files</span> $<span class="n">uri</span> =<span class="m">404</span>;
  }
}
</code></pre></div></div>

<h3 id="stage-cert">stage cert</h3>

<p>We run through one example in <code class="language-plaintext highlighter-rouge">staging</code> using <code class="language-plaintext highlighter-rouge">--register-unsafely-without-email --agree-tos </code>. This confirms to us letsencrypt ability to generate a valid cert for the site, and also guards against quotas.</p>

<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code>docker run <span class="nt">-it</span> <span class="nt">--rm</span> <span class="se">\</span>
<span class="nt">-v</span> <span class="nv">$data_path</span>/ssl:/etc/letsencrypt <span class="nt">-v</span> <span class="nv">$data_path</span>/www:/var/www/certbot <span class="se">\</span>
certbot/certbot <span class="se">\</span>
certonly <span class="nt">--webroot</span> <span class="se">\</span>
<span class="nt">--webroot-path</span><span class="o">=</span>/var/www/certbot <span class="se">\</span>
<span class="nt">--register-unsafely-without-email</span> <span class="nt">--agree-tos</span> <span class="se">\</span>
<span class="nt">--staging</span> <span class="se">\</span>
<span class="nt">-d</span> <span class="nv">$domain</span> <span class="nt">-d</span> www.<span class="nv">$domain</span>
</code></pre></div></div>

<h3 id="prod-cert">prod cert</h3>

<p>we now run this finally command to get the actual cert</p>

<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code>docker run <span class="nt">-it</span> <span class="nt">--rm</span> <span class="se">\</span>
<span class="nt">-v</span> <span class="nv">$data_path</span>/ssl:/etc/letsencrypt <span class="nt">-v</span> <span class="nv">$data_path</span>/www:/var/www/certbot <span class="se">\</span>
certbot/certbot <span class="se">\</span>
certonly <span class="nt">--webroot</span> <span class="se">\</span>
<span class="nt">--webroot-path</span><span class="o">=</span>/var/www/certbot <span class="se">\</span>
<span class="nt">--email</span> <span class="nv">$email</span> <span class="nt">--agree-tos</span> <span class="nt">--no-eff-email</span> <span class="se">\</span>
<span class="nt">-d</span> <span class="nv">$domain</span> <span class="nt">-d</span> www.<span class="nv">$domain</span>
</code></pre></div></div>

<h3 id="prod-cert-renew">prod cert renew</h3>

<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code>docker run <span class="nt">-it</span> <span class="nt">--rm</span> <span class="nt">-v</span> <span class="nv">$data_path</span>/ssl:/etc/letsencrypt <span class="nt">-v</span> <span class="nv">$data_path</span>/www:/var/www/certbot <span class="se">\</span>
certbot/certbot <span class="se">\</span>
renew <span class="nt">--webroot</span> <span class="nt">--webroot-path</span><span class="o">=</span>/var/www/certbot
</code></pre></div></div>

<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code>docker <span class="nb">exec</span> <span class="nt">-it</span> nginx nginx <span class="nt">-s</span> reload
docker logs nginx <span class="nt">-f</span>
</code></pre></div></div>

<p>At this stage I could docker run with a shell command wrapped in sleep to keep validating the cert and auto renew it, but since there’ll have to be a subsequent post where I try to automate this, we’ll leave it as it for now.</p>

<h3 id="temp-automated-cert-renewal">Temp automated cert renewal</h3>

<p>I got tired of having to log in and do this, so this is a good way to fake cron on container optimized os..</p>

<p>Run docker, with entrypoint, and sleep..</p>

<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>docker run --name renew -d -v $data_path/ssl:/etc/letsencrypt -v $data_path/www:/var/www/certbot --entrypoint="/bin/sh" certbot/certbot -c 'trap exit TERM; while :; do certbot renew --webroot --webroot-path=/var/www/certbot; sleep 12h &amp; wait ${!}; done;' 
</code></pre></div></div>

<h2 id="issues">issues</h2>
<ul>
  <li><a href="https://community.letsencrypt.org/t/no-resolver-defined-to-resolve-ocsp-int-x3-letsencrypt-org-while-requesting-certificate-status-responder-ocsp-int-x3-letsencrypt-org/21427">no-resolver-defined-to-resolve</a></li>
  <li><a href="https://github.com/mysqljs/mysql/issues/1507">mysqljs auth error</a></li>
</ul>]]></content><author><name>Kenneth Massada</name></author><category term="ghost" /><category term="blog" /><summary type="html"><![CDATA[I've been using the blog platform ghost for hosting tadbit.cc, and now twdspodcast.com, this is a quick brain dump on how I host the blog. It's simply nginx reverse proxy, ghost platform backed by mysql, and now the reverse proxy gets it's cert from let's encrypt]]></summary></entry><entry><title type="html">Life after WiSci 17’</title><link href="https://blog.kmassada.com/follow-up-on-wisci-17/" rel="alternate" type="text/html" title="Life after WiSci 17’" /><published>2018-01-18T18:24:00+00:00</published><updated>2018-01-18T18:24:00+00:00</updated><id>https://blog.kmassada.com/follow-up-on-wisci-17</id><content type="html" xml:base="https://blog.kmassada.com/follow-up-on-wisci-17/"><![CDATA[<p>Over last summer, I’ve been fortunate to be part of all girls camp. I discussed my experience <a href="http://blog.kmassada.com/volunteering-wisci-17/">volunteering at WiSci 17’</a>, mostly highlighting the humbling moments where the girls in turn were an our inspiration in eyes of all they’ve had to overcome. As time passed, I had this unshakable feeling, the eternal dilemma when embarking on a journey for a cause:</p>

<ul>
  <li>Is this enough? Is the education we are providing something far removed from the challenges of the economy they reside in?</li>
  <li>Are we doing enough to set them up for success beyond the camp?</li>
  <li>Quite frankly, in introspection, am I the kind of inspiration they need? How to be an ally to let them be inspired by the wonderful women of the camp?</li>
  <li>How are our stories taken by the girls, is perseverance the moral? or leaving their comfort zone? Could that have a negative impact in their communities?</li>
</ul>

<p>As I’ve asked myself these questions I’ve also found answers, not in the hypothetical or false correlation sense, but seeking data and concrete examples on how our help has been changing these girl’s life.</p>

<p>Is this enough? The State Department has embarked <a href="https://www.state.gov/r/pa/prs/ps/2018/01/277437.htm">one more time</a> on the adventure, the site of the camp will be in Windhoek this year. Consistency and continuity in this project will ensure that we give a lot of the same opportunities to young girls from various backgrounds. Just doing it again affect several young girls in the same communities or different communities this time, making it a norm to aspire to STEAM and broadening the impact of those who also can be affected by this.</p>

<p>Are we doing enough to set them up for success? We’ve been keeping close contact with the headmasters of the school, the program has a lot of pedagogical follow ups. My worries are of one who is only a participant, but from the organizers philosophy <a href="https://girlup.org/our-success/#sthash.O3Q0rO5h.tUAxYBlS.dpbs">a lot</a> went through this. The young girls are close to graduating highschool, it’s the perfect age to inspire a change in career path. The follow ups from their schools is ensuring continuity. The application process requires a mentor. That mentor is also on the path to follow ups, so they girls are not left alone feeling helpless.</p>

<p>The sub-Saharan African pedagogy on education is different. You don’t have to look <a href="https://www.youtube.com/watch?v=qc7J8CKfzaI">far</a> online to find skits and caricatures of what it is to be educated in a traditional African Family and the importance on education, but you also don’t have to look too far to see the gap between girls and boys education. From 2000-2015 <a href="http://unesdoc.unesco.org/images/0023/002348/234809E.pdf">UNESCO Global Education for All Global Monitoring Report</a>, No country in sub-Saharan Africa has
achieved gender parity in both primary
and secondary education, according to <a href="https://en.unesco.org/gem-report/sites/gem-report/files/SSA_Press_Release_English_Gender_Report2015.pdf">UNESCO Press Release</a> in 2015.</p>

<p>My inspiration to become an engineer comes from a lot of various factors, but one of the anecdotes that touches me the most is a social studies professor I once had who simply said: “You can do it!”. I’ve noticed in my journey for this answer is to keep putting at the forefront <strong>Lynn, Caroline, Astri, Adrienne, Lena, Darah, Urpi, Shariffa, Mayra, Kate, Lisa, Kim and Dei.</strong> These and many more are their inspiration, being there to remind them is amongst others, one thing I can do.</p>

<p>This is a “reportage documentary”, called “the Film” that answers all the questions I’ve been having about this, but most essentially the question that weighs the heaviest on me: What is the moral? What are these girls taking away from this. In this docu-series, you’ll see the genuine expression of the girls feeling through the camp, the lessons they’ve learned, learning moments for us and more. Stay tuned for my thoughts on Windhoek 2018!</p>

<iframe src="https://player.vimeo.com/video/243140512" width="640" height="360" frameborder="0" webkitallowfullscreen="" mozallowfullscreen="" allowfullscreen=""></iframe>
<p><a href="https://vimeo.com/243140512">&quot;I AM&quot; -  (File Title: RASCONA-OFFICIAL MALAWI FINAL MASTER h264)</a> from <a href="https://vimeo.com/user72872448">RASCONA PROD</a> on <a href="https://vimeo.com">Vimeo</a>.</p>]]></content><author><name>Kenneth Massada</name></author><category term="wisci" /><summary type="html"><![CDATA[Over last summer, I've been fortunate to be part of all girls camp. I discussed my experience [volunteering at WiSci 17'](http://blog.kmassada.com/volunteering-wisci-17/), here's an account of life after the camp]]></summary></entry><entry><title type="html">Volunteering in Malawi at WiSci 17’</title><link href="https://blog.kmassada.com/volunteering-wisci-17/" rel="alternate" type="text/html" title="Volunteering in Malawi at WiSci 17’" /><published>2017-08-18T18:24:00+00:00</published><updated>2017-08-18T18:24:00+00:00</updated><id>https://blog.kmassada.com/volunteering-wisci-17</id><content type="html" xml:base="https://blog.kmassada.com/volunteering-wisci-17/"><![CDATA[<blockquote>
  <p>WiSci Malawi brought together 100 high school girls from the African continent and the US for a three weeks long camp. The goal of the camp was to empower them to grow their leadership potential, and build camaraderie and networks that will propel them to new opportunities in their desired fields. The theme of WiSci 2017 is using technology to create a safer, more secure world, with a focus on preventing gender-based violence. WiSci partners this year included the UN Foundation’s Girl Up campaign, Intel Corporation, and Google.</p>
</blockquote>

<h1 id="malawi-and-must">Malawi and MUST</h1>
<p>The camp took place at MUST ‘Malawi University of Science and Technology’ campus in the Blantyre region in southern Malawi. The University is built in a somewhat rural area, Blantyre and it’s immediate surroundings is dominated by various land elevations. The campus, though, central to the elevations, causing sunrises and sunset to occur at high points.</p>

<figure class="third">
	<img src="/assets/images/wisci-2017/IMG_20170801_172110.jpg" />
	<img src="/assets/images/wisci-2017/IMG_2815.jpg" />
	<img src="/assets/images/wisci-2017/IMG_2906.jpg" />
	<figcaption>The campus and it's surroundings.</figcaption>
</figure>

<h1 id="the-girls-and-the-curriculum">The girls and the curriculum</h1>
<p>I was part of the Google staff. Our mission was to introduce computer science to the girls. Our curriculum had two parts: first, android development using MIT App Inventor, then arduino modules with wireless communications via bluetooth, tactile buttons and LED lights. The girls arrived with various backgrounds, some having admitted to have never touched a computer before. Others were already self-starters, here at the camp to make themselves more marketable. Our curriculum goal was to survey key concepts like variables, conditionals and sequence of executions, while keeping the girls engaged. One of the challenges in software education is translating concepts into something tangible, this is where the arduino modules help with a straight translation of a button press to an action such as a light on, or a sound.</p>

<figure class="third">
	<img src="/assets/images/wisci-2017/IMG_0917.jpg" />
	<img src="/assets/images/wisci-2017/IMG_2818.jpg" />
	<img src="/assets/images/wisci-2017/IMG_20170804_171739.jpg" />
	<figcaption>Classes at MUST.</figcaption>
</figure>

<h1 id="office-hours">Office Hours</h1>
<p>On top of the regular instructions, each instructors played the role of counselor during office hours, talking about our journeys in tech, empowering the girls to challenge themselves. It was a very humbling experience hearing the journeys of these young girls, the hurdles they’ve faced. As a male instructor, I was there in a role of ally and supporter, coming myself from a small country I could relate to the girls from several of the African countries represented. I had the unique opportunity to watch these impressive ‘Boss Ladies’ from the participating partners organizations and the camp itself inspire the girls, showing them that NASA Astronaut and Airline Pilot are attainable with instruction, skill, networking and mentorship.</p>

<h1 id="the-damore-memo">The ‘Damore Memo’</h1>
<p>This camp happened shortly before the memo was released. This, engineering and skilled trade hours dedicated to instruction, logistics, donations and so forth, more than never shows how education and instruction matters to the various companies invested in the camp. It does not matter on what side of the debate you sit on. It is clear that regardless of who they are, our youth need inspiration, and need choices, this camp did just that. GirlUp uses the term STEAM, with the emphasis on the needs in the Arts as well. Beyond, instruction of these young girls in the sciences and art, especially in Malawi, produces a diversity in skill, thought and leadership, with the hope to produce an immediate impact on the already rising trend of dynamic entrepreneurs, scientists and artists the African continent produces.</p>

<p>https://www.instagram.com/p/BXV5G6Hj4yv/</p>

<h1 id="african-experience">‘African’ Experience</h1>
<p>In addition to the instruction, and very rigorous program. The girls and the staff had the opportunity to participate in various extra curricular activities. The team I was involved with participated in a Safari, and another team went on a hike with the girls. Enriching the experience for these young women. The 100 girls were also divided by countries, taking part in cultural nights, presenting their cultures in various expression of art, music, storytelling and theater. On the organizer’s invitation, we’ve also had the privilege to listen to Zathu Band, a local Malawian part band, part radio production, in what was a very energetic concert with full crowd participation.</p>

<figure class="third">
	<img src="/assets/images/wisci-2017/IMG_20170803_113049.jpg" />
	<img src="/assets/images/wisci-2017/IMG_20170803_115147.jpg" />
	<img src="/assets/images/wisci-2017/MVIMG_20170803_105624.jpg" />
	<figcaption>Various excursions in Malawi.</figcaption>
</figure>

<h1 id="only-1-spark">Only 1 spark</h1>
<p>Coming from a small place, Togo isn’t the technological giant you’d expect, but all it takes is a spark to build interest, gain perspective and try something different. Despite the very scientific approach to a camp like this: defining impact on a community, studying trends in education, the strategic placement of the camp in Malawi and the careful selection of the participating nations, the biggest impact is the untold stories of the camp. The girls had a rigorous camp, building resilience and collaboration are very key, but also the very simple moment where they’ve learned they can or were inspired by someone is key. I know, I have, and I dearly wish them the same.</p>]]></content><author><name>Kenneth Massada</name></author><category term="wisci" /><summary type="html"><![CDATA[WiSci Malawi brought together 100 high school girls from the African continent and the US for a three weeks long camp. Here is my account at this camp.]]></summary></entry><entry><title type="html">Docker Lessons Learned: Hour 12</title><link href="https://blog.kmassada.com/projects/docker-primer/" rel="alternate" type="text/html" title="Docker Lessons Learned: Hour 12" /><published>2016-03-27T15:31:00+00:00</published><updated>2016-03-27T15:31:00+00:00</updated><id>https://blog.kmassada.com/projects/docker-primer</id><content type="html" xml:base="https://blog.kmassada.com/projects/docker-primer/"><![CDATA[<p>Aside from it being one of the hottest trends in tech today. I truly started looking at docker for <strong>practical</strong> reasons. I had a node in Digital ocean. On that node I run all my private projects, which vary from Node JS api listener for git post hooks to full stack web apps.</p>

<p>Recently I started getting notifications:</p>

<p><img src="https://blog.kmassada.com/assets/images/Screenshot_2016-03-28_10.06.2.png" alt="New Relic Alert" /></p>

<h3 id="what-it-may-be">what it may be?</h3>

<p>I have seen this long coming, and have been trying to prepare for it.</p>

<ul>
  <li>looked at <a href="https://gist.github.com/kmassada/578bdb2674624a40f18d">building my chef server </a>, to spin smaller instances, with a common base</li>
  <li>looked at <a href="https://github.com/kmassada/ansible">my own ansbile</a> to deploy projects.</li>
  <li>looked at capistrano for smaller projects</li>
  <li>looked at <a href="https://gist.github.com/kmassada/f3d635fb1d4b8219778d">Libvirt and Quemu</a> to drive vagrant. =.=</li>
</ul>

<h3 id="myths-design">myths! design!</h3>

<p>Let me start by debunking a few myths. The prime reason I chose containers was not solely due to cpu/memory utilization. <strong>This is not a tool problem, this is an architecture and a resource utilization paradigm.</strong></p>

<p>However,</p>

<ul>
  <li>re-architecting my applications in a way I could apply caps on each versus letting them all run wild on the system,</li>
  <li>having a fair separation in the libraries, versus using tech that isolate runtime environments,</li>
  <li>Not tying myself to nodes. Handling my nodes with the same maniability as I treat code,</li>
</ul>

<p>were the reasons I decided to use docker,</p>

<h3 id="the-right-way-things-i-wish-i-knew-first">the right way (things I wish I knew first)</h3>
<p>I jumped into the subject matter with a hast. Deadline: 12 hrs over the weekend to migrated all the projects. I learned very fast,</p>

<ul>
  <li>Like VMs, you need to specify autostart when a container is deployed</li>
  <li>Once a container deployed it is not easy to change it’s “state”. can’t re-specify <code class="language-plaintext highlighter-rouge">--start=always</code></li>
  <li>Prepare for containers to be truly maniable, killed, respun, and re-killed. redeploying a container with 0 downtime looks like this: <strong>deploy second container, switch nginx to another port, delete old container, docker run again. switch back nginx.</strong></li>
  <li><code class="language-plaintext highlighter-rouge">iptables.</code> dodged it like I do, yes brush up on it.</li>
  <li><code class="language-plaintext highlighter-rouge">Docker-compose</code> deploys containers using configuration files. After each container deploy code looks like below, you’d regret.</li>
</ul>

<p>docker run –name gelf -p 127.0.0.1:12201:12201/udp –link elasticsearch:elasticsearch -v /docker/gelf:/config-dir  –log-driver=gelf –log-opt gelf-address=udp://127.0.0.1:12201 –log-opt tag=”{{.ImageName}}/{{.Name}}/{{.ID}}” –restart=always  -d logstash  logstash -f /config-dir/gelf.conf</p>

<ul>
  <li>
    <p><code class="language-plaintext highlighter-rouge">DOCKER_OPTS</code> I have not fully figured it out on centos, but in ubuntu, you can set a series of OPTS that will get appened to your docker runs.</p>
  </li>
  <li><code class="language-plaintext highlighter-rouge">PROXY or vhosts</code>, because I have a series of services that serve on the same external ports. Using nginx as a proxy, that listens on that port, but with a different <code class="language-plaintext highlighter-rouge">server_name</code></li>
  <li><code class="language-plaintext highlighter-rouge">SWARM</code>, containers on their own are not clustered. Swarm resolves that problem. Don’t be lame Rob Lowe. Think swarm before anything.</li>
  <li><code class="language-plaintext highlighter-rouge">EXPOSED</code> is your door into the container either interacting with other nodes or your host. learn it.</li>
  <li>Volumes, your containers can share host resources, or if you catch it early, external resources. look into the volumes, and build solutions around deploying with or without.</li>
  <li>Honorable mention to: <code class="language-plaintext highlighter-rouge">Kitematic</code>. If you develop locally just freaking love it already. Its pretty on MAC.</li>
</ul>

<h3 id="school-of-thought">school of thought</h3>

<p>This gets it’s own section because it is important, and overshadows all the things previously mentioned. Containers are NOT VMs. At this point you are nodding, but take a deep breath, and think about it for a second.</p>

<p><img src="https://blog.kmassada.com/assets/images/containers_vs_vms.png" alt="containers vs vms" /></p>

<h3 id="the-run-down">The Run Down</h3>
<p>The big picture: With Docker create an ELK stack that monitors hosts and containers</p>

<p>The Details: Deploy a set of docker containers.</p>

<p>1- elasticsearch with logs stored on host.</p>

<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>docker run --name elasticsearch  -p 127.0.0.1:9200:9200  -v "/docker/elasticsearch/data":/usr/share/elasticsearch/data -p 127.0.0.1:9300:9300 --restart=always  -d elasticsearch
</code></pre></div></div>

<p>2- deploy a kibana dashboard that allows to search elasticsearch node.</p>

<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>docker run --name kibana -p 127.0.0.1:5601:5601 --link elasticsearch:elasticsearch --restart=always  -d kibana
</code></pre></div></div>

<p>3- deploy logstash with gelf (listens to port and sends data it receives to elastic search)</p>

<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>docker run --name gelf -p 127.0.0.1:12201:12201/udp --link elasticsearch:elasticsearch -v /docker/gelf:/config-dir --restart=always  -d logstash  logstash -f /config-dir/gelf.conf
</code></pre></div></div>

<p>4- configure containers to talk to gelf
re-deploy each container with the following</p>

<p>–log-driver=gelf –log-opt gelf-address=udp://127.0.0.1:12201 –log-opt tag=”{{.ImageName}}/{{.Name}}/{{.ID}}”</p>

<p>/docker/gelf/gelf.conf</p>

<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>input {
  gelf {
     host =&gt; "0.0.0.0"
     port =&gt; "12201"
     type =&gt; "gelf"
  }
}

output {
  stdout { codec =&gt; rubydebug }
  elasticsearch {
    hosts =&gt; ["elasticsearch:9200"]
  }
}
</code></pre></div></div>

<p>5- Docker daemon</p>

<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>sudo mkdir /etc/systemd/system/docker.service.d
sudo vi /etc/systemd/system/docker.service.d/docker.conf
</code></pre></div></div>

<p>docker daemon options</p>

<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>/usr/bin/docker daemon -H fd:// --log-driver=gelf --log-opt gelf-address=udp://127.0.0.1:12201
</code></pre></div></div>
<p>6- nginx configured for proxy</p>

<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>server {
    listen 80;

    server_name kibana.domain.com;

    auth_basic "Restricted Access";
    auth_basic_user_file /etc/nginx/htpasswd.users;

    location / {
        proxy_pass http://localhost:5601;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection 'upgrade';
        proxy_set_header Host $host;
        proxy_cache_bypass $http_upgrade;
    }
}
</code></pre></div></div>
<p>7- the result:</p>

<p><img src="https://blog.kmassada.com/assets/images/Screenshot_2016-03-28_11.59.40.png" alt="Hour 12" /></p>]]></content><author><name>Kenneth Massada</name></author><category term="projects" /><category term="docker" /><summary type="html"><![CDATA[My take on docker and how it helps with my personal projects]]></summary></entry><entry><title type="html">Docker Lessons Learned: 3 months in</title><link href="https://blog.kmassada.com/projects/docker-sequel/" rel="alternate" type="text/html" title="Docker Lessons Learned: 3 months in" /><published>2016-03-27T15:31:00+00:00</published><updated>2016-03-27T15:31:00+00:00</updated><id>https://blog.kmassada.com/projects/docker-sequel</id><content type="html" xml:base="https://blog.kmassada.com/projects/docker-sequel/"><![CDATA[<p>In my primer my usage was very limited, and there are a few questions i’ve been able to answer and I’d like to share the top ones. These I think are questions beginners itch to answer as they become intermediates.</p>

<h2 id="backing-up-image">Backing up image</h2>
<p>this will create data.zip of /usr/app/data and back it up at /data/backup</p>

<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>docker run --volumes-from data-container -v $(pwd):/usr/app ubuntu zip -r /usr/app/data /data/backup
</code></pre></div></div>

<h2 id="bash-into-image">Bash into image</h2>
<p>run a bash session straight into a container, where ID=container-name</p>

<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>docker run -it $ID bash
</code></pre></div></div>

<h2 id="restart-policy">Restart Policy</h2>
<p>I’ve always thought, running a container on a daemon will make it persist on error, but it doesn’t you have to set always restart.</p>

<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>docker run --restart=always -d redis /bin/bash
</code></pre></div></div>

<h2 id="security">SECURITY</h2>
<p>add some boundaries on what can and cannot be done on a container, because containers do offer a security woe when accessible from shell, it is hard to restrict from shell’s access</p>

<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>--cap-add and --cap-del
</code></pre></div></div>

<h2 id="nginx-proxy">nginx proxy</h2>
<p>jwilder nginx proxy allows you to, using environment variable VIRTUAL_HOST set on container, using docker remote events api, it can register any new domain you need for reverse proxy.</p>

<p>simply run the nginx container attached to port 80, and still run your containers as normal, it should pick your domains up and register them.</p>

<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>docker run -d -p 80:80 -v /var/run/docker.sock:/tmp/docker.sock jwilder/nginx-proxy
</code></pre></div></div>

<h2 id="cross-container-ipcommunication">cross-container IP/communication</h2>
<p>when in the same docker-compose, containers whithin the compose are accessible by either service name or container name automagically, however I had the use case where I like to keep a compose with every project, so I had a Node APi backed by Mongo, and that had it’s own docker-compose, but also had an angular client, with it’s own compose file. the way around is to create <code class="language-plaintext highlighter-rouge">--links</code> or <code class="language-plaintext highlighter-rouge">external-links:</code> in docker-compose. referencing other containers by <code class="language-plaintext highlighter-rouge">name:alias</code>, this alias then can be used in the code.</p>

<p>Or you could simply attempt to get a container’s IP by running</p>

<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>docker inspect --format='\{\{.NetworkSettings.IPAddress}}'  $ID
</code></pre></div></div>

<h2 id="labels-and-search">labels and search</h2>

<h2 id="registry">Registry</h2>

<h3 id="tag-and-push">tag and push</h3>

<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>docker tag [-f|--force[=false] IMAGE [REGISTRYHOST/][USERNAME/]NAME[:TAG]
docker push [REGISTRYHOST/][USERNAME/]NAME[:TAG]
</code></pre></div></div>]]></content><author><name>Kenneth Massada</name></author><category term="projects" /><category term="docker" /><summary type="html"><![CDATA[Tips and Tricks I've acquired after using docker for close to 3 months.]]></summary></entry><entry><title type="html">Projects</title><link href="https://blog.kmassada.com/projects/projects/" rel="alternate" type="text/html" title="Projects" /><published>2016-03-02T11:34:00+00:00</published><updated>2016-03-02T11:34:00+00:00</updated><id>https://blog.kmassada.com/projects/projects</id><content type="html" xml:base="https://blog.kmassada.com/projects/projects/"><![CDATA[<aside class="sidebar__right">
<nav class="toc">
    <header><h4 class="nav__title"><i class="fas fa-file-alt"></i> Projects</h4></header>
<ul class="toc__menu" id="markdown-toc">
  <li><a href="#voltbolt" id="markdown-toc-voltbolt">Voltbolt</a></li>
  <li><a href="#nodejs-api-server" id="markdown-toc-nodejs-api-server">NodeJS Api Server</a></li>
  <li><a href="#angular---typescript" id="markdown-toc-angular---typescript">Angular - TypeScript</a></li>
</ul>

  </nav>
</aside>

<h2 id="voltbolt">Voltbolt</h2>
<p><i class="fa fa-fw fa-github"></i><a href="https://github.com/slackerswithbots/presentation.git" title="https://github.com/slackerswithbots/presentation.git">Voltbolt</a></p>

<p>Build an interface that allow consumers to find volunteering opportunities next to them.</p>

<p><img src="https://raw.githubusercontent.com/slackerswithbots/presentation/master/images/Architecture.png" alt="voltbot architecture" /></p>

<h2 id="nodejs-api-server">NodeJS Api Server</h2>
<p><i class="fa fa-fw fa-github"></i><a href="https://github.com/kmassada/node.git" title="https://github.com/kmassada/node.git">NodeJS Api Server</a></p>

<h2 id="angular---typescript">Angular - TypeScript</h2>
<p><i class="fa fa-fw fa-github"></i><a href="https://github.com/kmassada/angular-typescript.git" title="Angular Typescript">Angular-Typescript</a></p>]]></content><author><name>Kenneth Massada</name></author><category term="projects" /><summary type="html"><![CDATA[survey to my projects]]></summary></entry></feed>