Regular expression

"$" are different in basic regular expression and extend regular expression, see https://stackoverflow.com/questions/38114132/why-does-dollar-not-match-literal-dollar-when-extended-regex-ere-option-is-use.

https://www.gnu.org/software/findutils/manual/html_node/find_html/posix_002degrep-regular-expression-syntax.html#posix_002degrep-regular-expression-syntax

• posix-egrep’ regular expression syntax: 
• The characters ‘^’ and ‘$’ always represent the beginning and end of a string respectively, except within square brackets. Within brackets, ‘^’ can be used to invert the membership of the character class being specified.
• ‘posix-basic’ regular expression syntax: 
  • The character ‘^’ only represents the beginning of a string when it appears:
  • At the beginning of a regular expression
  • After an open-group, signified by ‘\(’
  • After the alternation operator ‘\|’
  • The character ‘$’ only represents the end of a string when it appears:
  • At the end of a regular expression
  • Before a close-group, signified by ‘\)’
  • Before the alternation operator ‘\|’

Stack

• basic concept (from link)
• ghc only knows about packages that are installed. Installed packages live in package databases
• ghc-pkg is package tool from ghc and ghc know two database
• database is a directory, usually called package.conf.d, that contains a file for each package, together with a binary cache of the package data in the file package.cache
• global package database, which comes with your GHC installation, e.g. /usr/lib/ghc-6.12.1/package.conf.d.
• user package database private to each user, e.g. $XDG_DATA_HOME/ghc/arch-os-version/package.conf.
• LTS is may also not incomplete. 
• While we assume in general that LTS snapshots never change, there's nothing that technically prohibits that from happening. Instead, the complete version of that field is
  

  resolver:  size: 496662
    url: https://raw.githubusercontent.com/commercialhaskell/stackage-snapshots/master/lts/13/9.yaml
    sha256: 83de9017d911cf7795f19353dba4d04bd24cd40622b7567ff61fc3f7223aa3ea

• package in stack.yaml and package.yaml
• package.yaml:
• dependencies: package in LTS
• stack.yaml
• packages  : local package in local computer
• extra-deps: package from upstreams
• information in these two yaml just tells stack which packages are needed. Then stack download package and register them through ghc-pkg
• database name is decided by hash of snapshot and extra-deps
• .stack/snapshots/x86_64-linux-tinfo6/  => 02ca0c488465697d85d7ca7acaae14ad5743f138dce267e2ea1b93bd8c3bb3ab
• stack install xx will update local database and register xx package with ghc-pkg database, then these ghci could import these modules
• but database name may change if xx is new package in extra-deps
• global ghci: stack ghci out of any haskell program folder
• ~/.stack/global-project/stack.yaml
• package.yaml, cabal.project, global-project.cabal doesn't work
• stack exec (from link)
• The only issue is how to distinguish flags to be passed to stack versus those for the underlying program. Thanks to the optparse-applicative library, stack follows the Unix convention of -- to separate these, e.g.:
• stack exec --package stm -- echo I installed the stm package via --package stm
• "--package foo" can be used to force a package to be installed before running the given command

Windows

•  解决Windows10英文版中文字体难看、时大时小、中文比英文小等问题
• HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\FontLink\SystemLink
• 把 MSYH.TTC,Microsoft YaHei UI,128,96 MSYH.TTC,Microsoft YaHei UI放在Segoe UI，Tahoma，Microsoft Sans Serif的前面。
• see link1, link2, link3
• chrome installation
• chrome的proxy setting用的是windows system proxy setting，而公司的新电脑上system proxy是不可以更改的。在这种情况下，安装完chrome后是无法sync and install extension。可以用google-chrome --proxy-server="socks://localhost:8080"的方式用命令行来设置chrome proxy，后就可以sync and install extension。
• 公司电脑的政策是只有whitelist里的extension是可以安装的，可以用SwitchyOmega切换proxy
• WSL
• Windows 10 21H2里的wsl (Arch)默认还是wsl1，需要在setting ->Apps & feature -> Optional features -> more windows features里安装"windows subsystem for linux" (for WSL1) and "virtual machine platform" (for WSL2)
• Arch by default is WSL1，可以先把老的ext4.vhdx改名，再把wsl1 convert to wsl2，然后再把old hvdx的文件名改回ext4.vhdx及可。
• backup for new laptop
• onedrive, outlook,edge, chrome都是cloud based的，不用特殊备份
• netid里的老证书也可以从网上下载，不用备份
• backup chrome SwitchyOmega配置, .wslconfig in $user$ folder
• WSL(arch)需要把相关文件拷贝到onedrive里，安装完成后再copy出来。
• 其它非onedriver folder backup: log

Texlive

•  Installation with pacman 
• texlive-bin
• texlive-core
• texlive-fontsextra
• texlive-formatsextra
• texlive-langchinese
• texlive-latexextra
• texlive-pictures
• 
  
• texdoc
• install
• yay texlive-most-doc
• https://bbs.archlinux.org/viewtopic.php?id=241012
• "No texlive.tlpdb nor shipped tlpdb data found"
• texdoc (user manual) use TEXMFROOT/tlpkg/texlive.tlpdb (TEXMFROOT=/usr/share) but archlinux doesn't have it
• Solution 1: install extra AUR package: yay  texlive-tlpdb
• Solution 2: use local tlpdb
• tlmgr install --with-doc xxx
• font
• mkdir font folder: .local/share/fonts/
• install Ericsson and chinese font
• ./.local/share/fonts/ericsson/EricssonHilda-Medium.ttf
• ./.local/share/fonts/ericsson/EricssonTechnicalIcons-Regular.ttf
• ./.local/share/fonts/ericsson/EricssonHilda-Regular.ttf
• ./.local/share/fonts/ericsson/EricssonCapitalTT.ttf
• ./.local/share/fonts/ericsson/EricssonHilda-Light.ttf
• ./.local/share/fonts/ericsson/EricssonHilda-Bold.ttf
• ./.local/share/fonts/windows/simhei.ttf
• ./.local/share/fonts/windows/simkai.ttf
• ./.local/share/fonts/windows/simfang.ttf
• ./.local/share/fonts/windows/simsunb.ttf
• TFM
• tex/latex/pdflatex need TFM files for all fonts.
• XeTeX and LuaTeX have no need of TFM files for TrueType and OpenType fonts which contain the necessary metrics.
• see link
• tlmgr
• set in .bashrc
• alias tlmgr='TEXMFDIST/scripts/texlive/tlmgr.pl --usermode'
• use Tsinghua mirror
• tlmgr option repository https://mirrors.tuna.tsinghua.edu.cn/CTAN/systems/texlive/tlnet
• install texlive 2021 package when texlive 2022 is released
• tlmgr option repository https://ftp.tu-chemnitz.de/pub/tug/historic/systems/texlive/2020/tlnet-final
• tlmgr install xxx
• or tlmgr install xxx --repository https://ftp.tu-chemnitz.de/pub/tug/historic/systems/texlive/2020/tlnet-final
• repository
• default: default package repository found in the installation's TeX Live Package Database (a.k.a. the TLPDB, which is given entirely in the file tlpkg/texlive.tlpdb (for example ~/texmf/tlpkg/texlive.tlpdb)
• set in command line: tlmgr option repository https://mirror.ctan.org/systems/texlive/tlnet
• set in configuration file: make a permanent change, use option repository (see the "option" action).
• correct script error
• First, edit TEXMFDIST/scripts/texlive/tlmgr.pl and replace $Master ="$Master/../.."; with $Master = "${Master}/../../..";. The path prefix TEXMFDIST is not a predefined environment variable, it is defined below in this article. Replace it with the correct value.
• system configuration
• tlmgr conf
• TEXMFCONFIG=/home/hongy19/.texlive/texmf-config
• TEXMFDBS={!!/usr/local/share/texmf:/usr/share/texmf,!!/etc/texmf,!!/var/lib/texmf,!!/usr/share/texmf-dist}
• TEXMFDIST=/usr/share/texmf-dist
• TEXMFHOME=/home/hongy19/texmf
• TEXMFLOCAL=/usr/local/share/texmf:/usr/share/texmf
• TEXMFMAIN=/usr/share/texmf-dist
• TEXMFSYSCONFIG=/etc/texmf
• TEXMFSYSVAR=/var/lib/texmf
• TEXMFVAR=/home/hongy19/.texlive/texmf-var
• error in tlmgr check file
• (base) [hongy19@CN-00000988 ~]$ tlmgr check files
  Files present but not covered (relative to /home/hongy19/texmf):
  tex/generic/pgf/tikzlibrarytimeline/tikzlibrarytimeline.code.tex
  web2c/tlmgr-commands.log
  web2c/tlmgr.log
  tlmgr.pl: An error has occurred. See above messages. Exiting.
• (base) [hongy19@CN-00000988 ~]$ pacman -Qi texlive-core
  Name            : texlive-core
  Version         : 2021.61403-1
• (base) [hongy19@CN-00000988 ~]$ tlmgr --version
  tlmgr revision 60693 (2021-10-04 04:24:25 +0200)
  tlmgr using installation: /usr/share/texmf-dist/scripts/texlive/../../..
•   --package-logfile *file* "tlmgr" logs all package actions (install, remove, update, failed updates, failed restores) to a separate log file, by default "TEXMFSYSVAR/web2c/tlmgr.log". This option allows you to specify a different file for the log.
• --command-logfile *file* "tlmgr" logs the output of all programs invoked (mktexlr, mtxrun, fmtutil, updmap) to a separate log file, by default "TEXMFSYSVAR/web2c/tlmgr-commands.log". This option allows you to specify a different file for the log.
• when tlmgr running in user mode, log is stored in TEXMFHOME/web2c, not TEXMFSYSVAR/web2c

 if ($opts{"usermode"}) {
    $packagelogfile ||= "$::maintree/web2c/tlmgr.log";
  } else {
    $packagelogfile ||= "$texmfsysvar/web2c/tlmgr.log";
  }

• Kpathsea, TEXMFDBS and mktexlsr
• Kpathsea/kpsewhich use TEXINPUTS as search path
• Kpathsea/kpsewhich use ls-R database first, if no such database, it will search folder.
• ls-R database
• recommended way to create and maintain ‘ls-R’ is to run the mktexlsr
• TEXMFDBS used for ls-R
• TEXMFDBS ={!!/usr/local/share/texmf:/usr/share/texmf,!!/etc/texmf,!!/var/lib/texmf,!!/usr/share/texmf-dist}
• TEXMFDBS doesn't include TEXMFHOME which is searched for without ls-R files present.  kpsewhich could still find document in TEXMFHOME
• search document:  kpsewhich -all markdown.sty
•  TLPDB (texlive.tlpdb)
• tlpdb stands for tex live package data base, it is the main control file for the texlive package installation, it lists every package in texlive and whether you have it installed or not, which texlive mirror repository you use and lots of other details. The texlive installation seems broken if it is not there
• texlive.tlpdb on ctan is no use, the file is a local file that records which packages are installed on that machine it should be
• tlmgr init-usertree to generate texlive.tlpdb
• local texmf tree: TEXMFHOME
• https://www.ugr.es/~ftorralbo/blog/programming/local-texmf/
• mkdir /home/hongy19/texmf
• confirm TEXMFHOME path: kpsewhich -var-value=TEXMFHOME
• tlmgr init-usertree
• This creates *usertree*"/web2c" and *usertree*"/tlpkg/tlpobj", and a minimal *usertree*"/tlpkg/texlive.tlpdb".
• no need to run mktexlsr to create ls-R 
• latexmk
• copy .latexmkrc into ~
• markdown
• tlmgr install markdown
• xelatex --shell-escape markdown.tex
• add "\def\markdownOptionOutputDir{/tmp}" if Latex renders "Script file ./input.markdown.lua not found" error into pdf, see link

SSH

 

configuration on server side

on the /etc/ssh/sshd_config

• AllowGroups   wheel
• PasswordAuthentication no
• AuthenticationMethods publickey
• PermitRootLogin no

access server

put public key into server side ~/.ssh/authorized_keys and then use private key on your computer to access remote server.

SSH2_MSG_KEX_ECDH_REPLY

if encounter "SSH2_MSG_KEX_ECDH_REPLY" issue, try 

• KexAlgorithms
• ssh -oKexAlgorithms=ecdh-sha2-nistp521 server
• KexAlgorithms ecdh-sha2-nistp521 in ~/.ssh/config
• see link for detail and Key Exchange Method for background
• MTU
• see link for background
• ip link set mtu 1200

opensmtpd

 Installation

• pacman -S opensmtpd, opensmtpd-filter-dkimsign

Basic concept

mailbox

dovecot decide where to store email. it is /var/mail/user for mbox according to mail.local

dkim

dkim use private/public key to sign email. private key used to sign email and public key in DNS record to verify private key.

selector is name, you could choose what you want.

see smtpd.conf regardin how to use opensmtpd-filter-dkimsign to sign dkim. dkimproxy couls also be used but no formal/AUR package in Archlinux.

MAIL FROM and FROM

MAIL FROM is in envolope and mapping to Return-Path. "senders <users>" in smtp.config listen command is used to mapping user to email address, or example hongy19 -> hongy19@hongy19.net.

FROM is not in envelope and included into data. SMTP protocol doesn't care it. Mutt could change it. "masquerade" on smtp.config use to change it, for example hongy19 -> hongy19@hongy19.net

action and match

"When mail arrives, each “RCPT TO:” command generates a mail envelope. If an envelope matches any of a pre-designated set of criteria (using the match directive), the message is accepted for delivery. A copy of the message, as well as its associated envelopes, is saved in the mail queue and later dispatched according to an associated set of actions (using the action directive). If an envelope does not match any options, it is rejected. The match rules are evaluated sequentially, with the first match winning." according to smtpd.conf

For out-email (relay in action), need to relay only authorization email (through match auth), otherwise spam email will forwarded.

Checking queue

• smtpctl show queue

New vultr system

 Archlinux install

• install pacman-mirrorlist
• install ntp. systemtcl enable/start ntpd.service
• set timezone: ln -sf /usr/share/zoneinfo/Asia/Shanghai /etc/localtime
• locale: add LANG=en_US.UTF-8 into /etc/locale.gen. command "locale-gen "
• network configuration
• ip addr -> enp1s0
• /etc/systemd/network/20-wired.network
• systemctl enable/start systemd-networkd
• add user
• useradd --create-home –-groups wheel <yourusername>
• preset all systemctl unit
• systemctl preset-all (preset unit enable/disable according to /etc/systemd/system-preset)
• install yay for AUR package

SSH

• install fail2ban sshguard
• copy old fail2ban configuration file
• install openssh
• update /etc/ssh/sshd_config
• PasswordAuthentication no
• AuthenticationMethods publickey
• copy public key into /home/user/.ssh/authorized_keys

certbox

• install certbot certbot-nginx
• Certbot renew error: Account at does not exist.

Nginx

• install nginx
• copy old configuration
• firewall setting with iptables
• install uwsgi uwsgi-plugin-python
• copy old configuration
• use emperor mode and put all *.ini in /etc/uwsgi/vassals
• sudo systemctl enable uwsgi@emperor.service --now
• for my application
• pymoney: python-flask-httpauth python-flask python-matplotlib
• pyblog:  python-google-api-python-client python-iso8601

Dovecot

• install dovecot
• copy old configuration file
• firewall setting with iptables
• certification need to be ready with certbot

opensmtpd

• install s-nail, mutt
• install opensmtpd
• install opensmtpd-filter-dkimsign
• see link, link1, link2
• dkimproxy doesn't exist in Archlinux anymore, port 10027 doesn't work link
• copy old configuration
• firewall setting with iptables
• certification need to be ready with certbot
• port 25 is blocked by default on vultr to avoid spam and need ticket to open it. See link for "Network error on destination MXs" error

Archlinuxcn

• add archlinuxcn in pacman.conf, see link
• install archlinuxcn-keyring, otherwise you will see "signature from "xxx <xxx@build.archlinuxcn.org>" is unknown trust ", see link.