blog

Categories     Timeline     RSS

Concept2 PM5 with LaymansHex

For the fun of it, I decided to reimplement a basic version of pm5conv in LaymansHex and a bit of Bash - to parse workout data of a Concept2 PM5 monitor.

I basically took the data format description and translated it to LaymansHex definitions and Bash glue:

% for i in `ls | grep -E "(layhex|bash)"`; do echo "==> $i\n"; cat $i; echo; done
==> LogDataAccessTbl.bin.layhex

little endian
            : byte[offset*32]
Magic       : byte[1]
WorkoutType : uint8
            : byte[10]
NoSplits    : uint16
            : byte[2]
Offset      : uint16
            : byte[6]
Size        : uint16
Index       : uint16
            : byte[4]
Marker      : byte[32]

==> LogDataStorage.bin-Workout1.layhex

big endian
                    : byte[offset]
                    : byte[1]
                    : byte[1]
                    : byte[2]
                    : byte[4]
Timestamp           : byte[4]
UserID              : uint16
                    : byte[4]
RecordID            : uint8
                    : byte[3]
TotalDuration       : uint16
TotalDistance       : uint32
SPM                 : uint8
                    : byte[1]
SplitSize           : uint16
                    : byte[18]
                    : byte[splitNo*32]
SplitDistance       : uint16
SplitHeartRate      : uint8
SplitSPM            : uint8
                    : byte[28]

==> pm5conv.bash

#!/bin/bash

cmd="laymanshex"

accTable="LogDataAccessTbl.bin"
storage="LogDataStorage.bin"

# uses evil eval

function printNonEmpty {
  varname='$'"$1"
  eval "val=$varname"
  if [ "$val" != "" ]; then
      echo "$1=$val"
  fi
}

function printTimestamp {
  timestamp="0x$1"
  year="$(( ((timestamp & 0xFE000000) >> 25) + 2000 ))"
  day="$((   (timestamp & 0x01F00000) >> 20 ))"
  month="$(( (timestamp & 0x000F0000) >> 16 ))"
  hour="$((  (timestamp & 0x0000FF00) >> 8  ))"
  minute="$(( timestamp & 0x000000FF ))"
  printf "Date=%d-%02d-%02d %02d:%02d\n" $year $month $day $hour $minute
}

function printSplit {
  outputSplit="$($cmd -nopadding -fvar=offset=$1,splitNo=$2 $storage-Workout$3.layhex $storage 2> /dev/null)"
  status=$?
  if [ $status -eq 0 ]; then
    eval "$outputSplit"
    echo
    echo "Split $(($2+1))"
    echo "----------"
    printNonEmpty "SplitDuration"
    printNonEmpty "SplitDistance"
    printNonEmpty "SplitHeartRate"
    printNonEmpty "SplitSPM"
  fi
}

function printWorkout {
  echo "Workout $1"
  echo "============="
  outputHeader="$($cmd -nopadding -fvar=offset=$2,splitNo=0 $storage-Workout$3.layhex $storage 2> /dev/null)"
  status=$?
  if [ $status -eq 0 ]; then
    eval "$outputHeader"
    printTimestamp "$Timestamp"
    printNonEmpty "TotalDuration"
    printNonEmpty "TotalDistance"
    printNonEmpty "SplitSize"
    printNonEmpty "SPM"
    j=0
    while [ "$j" -lt "$NoSplits" ]; do
      printSplit $2 $j $3
      j=$((j+1))
    done
    echo
  fi
}

function printAll {
  i=0
  while : ; do
    offset=$(($i))
    output="$($cmd -nopadding -fvar=offset=$offset $accTable.layhex $accTable 2>/dev/null)"
    status=$?    
    if [ $status -ne 0 ]; then
      break
    else
      eval "$output"    
      printWorkout $Index $Offset $WorkoutType
    fi
    i=$((i+1))
  done
}

printAll

Output of pm5conv.bash:

% ./pm5conv.bash
Workout 1
=============
Date=2018-02-27 21:00
TotalDuration=962
TotalDistance=363
SplitSize=3000
SPM=29

Split 1
----------
SplitDistance=363
SplitHeartRate=0
SplitSPM=27

Workout 2
=============
Date=2018-02-28 20:41
TotalDuration=4208
TotalDistance=1559
SplitSize=3000
SPM=27

Split 1
----------
SplitDistance=1121
SplitHeartRate=0
SplitSPM=29

Split 2
----------
SplitDistance=438
SplitHeartRate=0
SplitSPM=26

Workout 3
=============
Date=2018-03-01 18:47
TotalDuration=12058
TotalDistance=4166
SplitSize=3000
SPM=28

Split 1
----------
SplitDistance=1056
SplitHeartRate=0
SplitSPM=29

Split 2
----------
SplitDistance=1060
SplitHeartRate=0
SplitSPM=28

Split 3
----------
SplitDistance=1007
SplitHeartRate=0
SplitSPM=28

Split 4
----------
SplitDistance=1024
SplitHeartRate=0
SplitSPM=29

Split 5
----------
SplitDistance=19
SplitHeartRate=0
SplitSPM=0

[...]

Sequences in LaymansHex

I have spent some time thinking about handling sequences and in-file offsets with LaymansHex. I decided on two things:

  1. LaymansHex is not and should not be used as a “proper” binary parser. It is a quick-and-dirty tool to read and set some values.
  2. Binary sequences are implemented in multiple ways and adding every way to the file description format is tedious and makes the use complex.

Both points led me to believe, that it’s better to handle sequences and offsets with multiple calls to LaymansHex and variable byte field sizes in the format definition to allow for offsets and sliding windows. I implemented this in commit 279429d0bff0681d533b23c260249361078d0a78

I added a minimal example to the of the README.

theHunter COTW player file format

theHunter: Call of the Wild is a hunting simulator, which - unfortunately - has some characteristics of a typical first-person shooter, namely missions, rewards and locked equipment. Since this prevents actual hunting, I thought it would be interesting to analyse the file format that saves level, XP, cash and so on.

I wrote a fairly generic tool called laymanshex that takes a partial file description and a binary file as input and outputs the values. Values can be changed with the ‘-set’ argument.

For example:

% ./laymanshex thp_player_profile_adf.layhex thp_player_profile_adf
           Level = 19
              XP = 20235
     SkillPoints = 0
      PerkPoints = 0
SkillPointsSpent = 7
 PerkPointsSpent = 6
            Cash = 16640
      RifleLevel = 14
    HandgunLevel = 4
    ShotgunLevel = 4
        BowLevel = 1
      RifleScore = 1992
    HandgunScore = 382
    ShotgunScore = 417
        BowScore = 0

% ./laymanshex -set="Level=60,XP=100000" thp_player_profile_adf.layhex thp_player_profile_adf
Created backup thp_player_profile_adf.bak20200426231811

           Level = 60
              XP = 100000
     SkillPoints = 0
      PerkPoints = 0
SkillPointsSpent = 7
 PerkPointsSpent = 6
            Cash = 16640
      RifleLevel = 14
    HandgunLevel = 4
    ShotgunLevel = 4
        BowLevel = 1
      RifleScore = 1992
    HandgunScore = 382
    ShotgunScore = 417
        BowScore = 0

% ./laymanshex -set="Cash=0x7FFFFFFF" thp_player_profile_adf.layhex thp_player_profile_adf | grep Cash
            Cash = 2147483647

The partial file description for thp_player_profile_adf (COTW version 1.49) looks like this:

little endian
                 : byte[113]
Level            : int32
XP               : int32
SkillPoints      : int32
PerkPoints       : int32
SkillPointsSpent : int32
                 : byte[60]
PerkPointsSpent  : int32
                 : byte[60]
Cash             : int32
RifleLevel       : int32
HandgunLevel     : int32
ShotgunLevel     : int32
BowLevel         : int32
RifleScore       : int32
HandgunScore     : int32
ShotgunScore     : int32
BowScore         : int32

Go sources can be found in the laymanshex git repo and possible further file descriptions in the laymanshex-files git repo.

The file format of COTW saves has changed many times in the past and I assume it will again at some point in the future.

Receiver-side Cutoff

With the corona crisis comes an increased use of remote work tools, especially communication tools to substitute for face to face interaction. There has been a lot of discussion about security, privacy and the like - what I haven’t seen discussed much is the burden different tools put on the different parties involved. What I miss dearly in some tools is a Receiver-side Cutoff - a convenient way to hinder senders from sending further content. Not delay, not silently delete - hinder. Something every communication service should have.

I’ve had to use a matrix / riot.im instance - and I’m not a fan. Other people can invite me to rooms and direct chats, I can’t seem to limit this in a sensible way, and people can continue writing me even when I’m logged out. It’s like email and mailing lists, only worse: The gooey GUI further lowers the threshold for sending badly thought-out messages. A textual chat room is not a substitute for a stringently organized meeting. A textual direct chat is not a substitute for face to face interaction if one party can continue rambling when the other is absent.

The phone, intrusive as it is, at least limits the number of simultaneous senders to one. And it can conveniently be switched to not disturb. Email is a bad offender in this regard - ideally my company inbox should reject any internal message not authored by the chief executives outside my working hours. But matrix feels even worse.

[Update 2023-02-06:] The truly broken thing about this: The higher your perceived value to others, the bigger the asymmetry between number of senders and receivers. All forms of communication foster this to some degree, but most have some form of natural limitation. If you take away the limitation, most people will instinctively react by trying to not display any valuable skills or traits. Some will try to please everyone and inevitably fail. A few learn to set boundaries, but have to defend them with increasing demands on time and self-assertion. This is a shit experience for everyone.

Make X less annoying

Two new custom CSSes to get rid of some web mistakes:

Instagram:

body   { overflow: auto !important; }
.tHaIX { display: none !important; } /* no "please login" footer */
.RnEpo { display: none !important; } /* no login pop up */
._lz6s { display: none !important; } /* no login/register header */ 

Facebook:

._5hn6 { display: none !important; }             /* no login footer */
._1pfm { display: none !important; }             /* no like, share, send message */
._57dz { display: none !important; }             /* no side column */

I’ve also made a new git repo to track changes.

< Older

Newer >