Alles over…
Alles over…

P1-Monitor docker container for smart meters

A docker-based DSMR reader

This project is describing the porting of the P1-Monitor software running on Raspberry Pi to a container version to be able to run it on any Operating System. The original software is maintained from ztatz.nl

For those new to this subject, this software is intended to read information from you smart energy meter using the P1 port (a sort of serial port) using the DSMR protocol. A data package, called a telegram, is sent every 10 seconds (interval may differ) that contains data on usage on electricity and gas. You can use that data to get real-time insight in your usage. P1 monitor is software that visualises the data using graphs en retain the information in databases.

Since I’m running a Linux server already I didn’t want to start using a dedicated Raspberry Pi board for monitoring my smart meter. But the software provided by ZTATZ is only available as a complete system image running on a Raspberry Pi. That’s why I started converting that software to a container version suitable for running on any system with the help of docker, docker-compose, portainer or any system.
One other big benefit that I experienced is that I do not need to go through the extensive update process of exporting and importing data since everything is stored outside the container. Unless changes will be made to the databases of course but in that case you still use the export/import procedures.

The container image is available on Docker hub and I’m using GitHub to maintain the repository. The container version has the same functionality as the full version (although some options related to Operating System are not being used of course). Docker hub also holds the versions for other architectures so you can also run the container on ARM etc (such as Raspberry Pi) devices.

Connectors

Since I don’t have a serial port on my server (who does?) I used a standard USB to serial adaptor and modified a standard phone cable (for the RJ11 connector in the Smart meter) and connected it to the serial as below (note that RxD and TxD are swapped. Of course its even more easy to buy a standard ‘Smart meter USB cable‘ for 15 euro or so but it was weekend and I needed the cable now 🙂

P1-Monitor smart monitor cable connector

Install docker

For the ones not familiar with containers: Start with installing docker and docker-compose on your system. I selected docker-compose because it is easier to start, read and maintain a configuration. If you rather would like the use the docker command-line options an example can be found below.

Docker script

To start with docker-compose create a new directory and create a file named docker-compose.yml. Copy the contents of below docker-compose file example to that file and save it. (Note you keep the indentation with spaces!) If needed you can modify it using your own parameters such as the location of the data (in the example script in the subdirectory alldata). That subdirectory is for creating persistence; if the container is recreated or an update is available all data is preserved since it is being stored outside the container. If your server is rebooted the container will automatically be restarted.

Next you start your containerised P1-Monitor using the docker-compose up -d command. This way a small local network will be created and the container will be started in the background. After a short while (initially the container image needs to be downloaded from Docker hub) the P1-Monitor will be available on selected port. In below example it will be running on standard port 80 but you can easily change that by modifying the port mapping parameter from 80:80 to for instance 81:80 if you want the container to be available at port 81.

Similar is the devices section; it is mapping your local serial device to /dev/ttyUSB0 in the container

Docker-compose file example

version: '3.8'
services:

  p1monitor:
    hostname: p1mon
    image: mclaassen/p1mon
    ports:
      - 80:80
    volumes:
      - ./alldata/data:/p1mon/data
      - ./alldata/usbdisk:/p1mon/mnt/usb
      - ./alldata/mnt/ramdisk:/p1mon/mnt/ramdisk
    tmpfs:
      - /run
      - /tmp
    devices:
      - /dev/ttyUSB0:/dev/ttyUSB0
    restart: unless-stopped

Parameters

The docker-compose file supports below parameters:

  • ports
    mapping van ports from outside to inside. If you want to use port 81 to access your instance use the setting 81:80
  • volumes
    these are the directories being used to store the persistent data. In the example above subdirectory alldata will be automatically created (if started for the first time) and all collected information will be stored here.
  • devices
    mapping of your external to internal serial device. Adjustment is only needed in case you are using a different USB port on your host system.

Upgrade to latest version

If a new container is released (usually shortly after the original P1-Monitor software has been updated) you can stop the container instance using the docker-compose down command. Do not delete any files but issue the command docker-compose pull to retrieve the latest image from Docker hub. The pull command will download the latest image as specified in the docker-compose file. After the pull is complete start the container again using the docker-compose up command. All historic data will be reused (if you do not change the docker-compose configuration file of course).

Note that new container versions will not automatically update.

Migrate from original P1-Monitor instance

To migrate from the original P1-Monitor (the one you had running on the dedicated Raspberry Pi) it is easiest to export the data. To do so use the menu Settings (Instellingen) and select In- Export. In your new instance use the same menu to import the extracted data again.

Additional docker-compose parameters

Additional parameters can be passed to the container using the option environment. As example in above configuration adding the available parameters as shown below.

version: '3.8'
services:

  p1monitor:
    hostname: p1mon
    image: mclaassen/p1mon
    ports:
      - 80:80
    volumes:
      - ./alldata/data:/p1mon/data
      - ./alldata/usbdisk:/p1mon/mnt/usb
      - ./alldata/mnt/ramdisk:/p1mon/mnt/ramdisk
    tmpfs:
       - /run
       - /tmp
    devices:
       - /dev/ttyUSB0:/dev/ttyUSB0
    environment:
       - DISABLECPUTEMP=true # obsolete since v1.5.0b
       - PROXYPATH=p1mon
       - SOCAT_CONF="pty,link=/dev/ttyUSB1,rawer,group-late=dialout,mode=660 tcp:192.168.1.200:5523,retry=forever,interval=30"
    restart: unless-stopped

DISABLECPUTEMP

This option is obsolete since version 1.5.0b. The container start script will automatically check if CPU temperature is available.

Version 1.5.0a and earlier: On certain Operating Systems the CPU temperature is not available causing the WatchDog script to crash. Adding parameter DISABLECPUTEMP=true will disable this option. Default value of this option is set to false

PROXYPATH

Since I would like access to the data also when not at home I added a reverse proxy option. By adding parameter PROXYPATH=p1mon the configuration in the container is adjusted so you can use the subdirectory p1mon as a path in your url. Your outside url would be something like https://myhomenetwork.nl/p1mon.
Don’t forget to protect your reverse proxy (e.g. username/password) as the P1-Monitor version is not build to be internet facing.

SOCAT_CONF

Some time back a few users requested to have socat implemented in the image for remote reading of the P1 data to a virtual serial port. The example parameter above is reading information from a remote serial reader at 192.168.1.200 port 5523. When starting the container port ttyUSB1 is created and will be used in P1Monitor to read the data. Logging is available in /var/log/socat.log. Please adjust the configuration as needed.

Depending on the remote device you’re using socat parameters may differ. Check the information from the supplier (or software vendor) for the correct parameters. Users have noted for instance that they needed to use raw or raw,echo 0 instead of rawer and modify group-late to group to get the connection up.

Note that the devices section is obsolete when using socat as the script is creating a virtual serial port itself.

Accessing GPIO ports

To get access for the GPIO pins on Raspberry Pi devices (for instance to read water meter information), add below modifications to your docker-compose script. Some users reported to have the privileged: true parameter added to the p1monitor section (just below the image tag) to get access to the GPIO ports.

version: '3.8'
services:

  p1monitor:
    hostname: p1mon
    image: mclaassen/p1mon
    ports:
      - 80:80
    volumes:
      - ./alldata/data:/p1mon/data
      - ./alldata/usbdisk:/p1mon/mnt/usb
      - ./alldata/mnt/ramdisk:/p1mon/mnt/ramdisk
      - /sys:/sys
    tmpfs:
      - /run
      - /tmp
    devices:
      - /dev/ttyUSB0:/dev/ttyUSB0
      - /dev/gpiomem:/dev/gpiomem
    restart: unless-stopped

Using a ramdisk

The original software of P1 Monitor is using a ramdisk for writing to the databases. Reason is to avoid to many write actions to the SD cards (as they can become corrupted or something). Every now and then data from ramdisk is written to the filesystem. When running on a Linux server that is not a real issue as the file IO is not that much. If you still want to use a ramdisk you can use below modification in the docker-compose file. Remove the ramdisk from the volumes section and add it to the tmpfs section as shown below:

version: '3.8'
services:

  p1monitor:
    hostname: p1mon
    image: mclaassen/p1mon
    ports:
      - 81:80
    volumes:
      - ./alldata/data:/p1mon/data
      - ./alldata/usbdisk:/p1mon/mnt/usb
    tmpfs:
      - /run
      - /tmp
      - /p1mon/mnt/ramdisk
    devices:
      - "/dev/ttyUSB0:/dev/ttyUSB0"
    restart: unless-stopped

Storing logfiles outside container

When using the default configuration the logfiles are stored inside the container and visible via the P1-Monitor user interface or when logging into the container. If you want to have direct access to your logfiles you can store them outside the container by adding a volume such as ./alldata/log:/var/log/p1monitor in the volumes section.

Using docker command-line options

All examples in this blog are based on using docker-compose. Of course you can also use the standard docker command-line options. As noted earlier, more difficult to read but same functionality. This example, based on the first docker-compose example from above, will create a detached container named p1monitor listening on external http port 81, mapping /dev/ttyUSB0 to the same internal device and using/storing data in <your_data_dir>:

docker run -d -p 81:80 -h p1mon --name=p1monitor --tmpfs /tmp --tmpfs /run -v <your_data_dir>/data:/p1mon/data -v <your_data_dir>/usbdisk:/p1mon/mnt/usb -v <your_data_dir>/mnt/ramdisk:/p1mon/mnt/ramdisk --device /dev/ttyUSB0:/dev/ttyUSB0 --restart=unless-stopped mclaassen/p1mon

Running in kubernetes variants

Using below example its also possible to run P1Monitor container version on for instance Kubernetes/Rancher/K3s environments. The example is based on a deployment using /home/user/k3s/p1mon/alldata as persistant data directory and assuming your cluster is running on 192.168.10.10 and serviced on port 8081. The PROXYPATH environment variable is set as example. The deployment and service are using namespace p1monitor.

---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: p1monitor
  labels:
    app: p1monitor
  namespace: p1monitor
spec:
  replicas: 1
  selector:
    matchLabels:
      app: p1monitor
  template:
    metadata:
      labels:
        app: p1monitor
    spec:
      containers:
        - name: p1monitor
          securityContext:
            privileged: true
          env:
            - name: PROXYPATH
              value: power
          image: mclaassen/p1mon
          ports:
            - containerPort: 80
              protocol: TCP
          resources: {}
          volumeMounts:
             - mountPath: /dev/ttyUSB0
               name: ttyusb
             - mountPath: /p1mon/data
               name: p1monitor-data
             - mountPath: /p1mon/mnt/usb
               name: p1monitor-usbdisk
             - mountPath: /var/log/p1monitor
               name: p1monitor-log
             - mountPath: /run
               name: p1monitor-run
             - mountPath: /tmp
               name: p1monitor-tmp
             - mountPath: /p1mon/mnt/ramdisk
               name: p1monitor-ramdisk
      restartPolicy: Always
      volumes:
        - hostPath:
            path: /dev/ttyUSB0
          name: ttyusb
        - hostPath:
            path: /home/user/k3s/p1mon/alldata/data
          name: p1monitor-data
        - hostPath:
            path: /home/user/k3s/p1mon/alldata/usbdisk
          name: p1monitor-usbdisk
        - hostPath:
            path: /home/user/k3s/p1mon/alldata/log
          name: p1monitor-log
        - emptyDir:
            medium: Memory
          name: p1monitor-run
        - emptyDir:
            medium: Memory
          name: p1monitor-tmp
        - emptyDir:
            medium: Memory
          name: p1monitor-ramdisk
---
apiVersion: v1
kind: Service
metadata:
  name: p1monitor
  labels:
    app: p1monitor
  namespace: p1monitor
spec:
  selector:
    app: p1monitor
  type: NodePort
  externalIPs:
  - 192.168.10.10
  ports:
   - port: 8081
     targetPort: 80

Version information

This is the list of changes for the different container versions. the latest tag (if you omit the tag latest will be used) will take you to the newest version. If you would like to use a different version add it to the image tag in the compose file. E.g. If you want use container version 1.4.1b modify the image tag to mclaassen/p1mon:1.4.1b

  • 1.6.0
    Updates to latest released version 1.6.0
  • 1.5.0d
    Fixed issue with rights on data directory and updated base image
  • 1.5.0c
    Fix issues related to ARM devices and GPIO port assignments
  • 1.5.0b
    Fixed minor issues, automated DISABLECPUTEMP variable (obsolete now)
  • 1.5.0a
    Updated system, completed socat support
  • 1.5.0 
    Updates to latest released version 1.5.0
  • 1.4.1b 
    Changed to running as p1mon, fixed issues, updated vulnerable packages
  • 1.4.1a 
    Fixed issue tariff status as described by author
  • 1.4.1 
    Based on the 1.4.1 image of ZTATZ. Also added reverse proxy capability
  • 1.4.0 
    Based on the withdrawn image of ZTATZ. Fixed 1 second issue. Image was withdrawn by the original author but due to the nature of docker container most bugs are not applicable here.
  • 1.3.1d 
    Fixed temperature error in VM using environment variable. Modified cron
  • 1.3.1c 
    Fixed import/export functions as well as missing ping utility
  • 1.3.1b 
    added health check function. Container will show health status and restart on failure
  • 1.3.1a 
    Updates fixing some vulnerabilities in python packages and dpkg
  • 1.3.1 
    Update to latest released version 1.3.1
  • 1.3.0 
    Update to latest released version 1.3.0
  • 1.2.0 
    Initial released version 1.2.0

More information

For more information on P1-Monitor such as latest updates, news and forum take a look at ZTATZ, GitHub, P1Monitor forum en op Docker Hub

Een reactie plaatsen

Het e-mailadres wordt niet gepubliceerd.

33 gedachten over “P1-Monitor docker container for smart meters”