Percona支持RHEL/CentOS/Debian/Ubuntu等GNU/Linux發行版,在Installing Percona Server from Repositories中列出了具體支持的發行版本,但並未顯式地說明Percona Server for MySQL的每一個釋出版本(如5.6,5.7)具體支持哪些GNU/Linux發行版本。

本文通過Percona Software Downloads提取各個可供下載的Percona版本的下載頁鏈接,再通過下載頁的HTML代碼提取各版本具體支持的GNU/Linux發行版,最後通過Shell Script代碼實現。

重要Percona官方已經停止對RHEL 5Ubuntu 12.04 LTS的支持,詳細說明見Platform End of Life (EOL) Announcement for RHEL 5 and Ubuntu 12.04 LTS

RHEL 5 was EOL as of March 31st, 2017 and Ubuntu 12.04 LTS was end of life as of April 28th, 2017.

Requirement

提取每一個Percona所支持的GNU/Linux發行版本具體支持的Percona版本列表

  1. Percona具體支持哪些GNU/Linux發行版;
  2. 每一個GNU/Linux發行版具體支持Percona哪些版本;

## Analysis
通過分析Percona Software Downloads頁面,提取Percona目前提供下載的版本號及對應的下載頁鏈接。

1
2
3
4
5
6
curl -fsL https://www.percona.com/downloads/ | sed -r -n '/A drop-in replacement for MySQL/,/<\/div>/{/<a/{[email protected]* href="([^"]*)".*>Download ([[:digit:].]+).*@\2 https://www.percona.com\[email protected];p}}'

# 5.7 https://www.percona.com/downloads/Percona-Server-LATEST/
# 5.6 https://www.percona.com/downloads/Percona-Server-5.6/LATEST/
# 5.5 https://www.percona.com/downloads/Percona-Server-5.5/LATEST/
# 5.1 https://www.percona.com/downloads/Percona-Server-5.1/LATEST/

通過下載頁鏈接,提取具體支持的GNU/Linux發行版。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
# 5.7
curl -fsL -x socks5h://101.100.176.53:54609 https://www.percona.com/downloads/Percona-Server-LATEST/ | sed -r -n '/Select Software Platform/{[email protected]/@[email protected];[email protected]<\/option>@\[email protected];p}' | sed -r -n '[email protected]*/([^"]*)"[[:space:]]*>.*@\[email protected];/binary|source|select|^$/d;p'

# wheezy
# jessie
# stretch
# rhel6
# rhel7
# precise
# trusty
# xenial
# yakkety
# zesty


# 多行轉換爲一行
sed -r ':a;N;$!ba;[email protected]\[email protected] @g;'

# wheezy jessie stretch rhel6 rhel7 precise trusty xenial yakkety zesty

Code Snippets

通過兩種方式實現:while循環和parallel並行操作。

Original

利用while循環,較爲耗時

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
#!/usr/bin/env bash

download_tool=${download_tool:-'curl -fsL'}
version_list_info=$(mktemp -t tempXXXXX.txt)

# - version|distro_list
$download_tool https://www.percona.com/downloads/ | sed -r -n '/A drop-in replacement for MySQL/,/<\/div>/{/<a/{[email protected]* href="([^"]*)".*>Download ([[:digit:].]+).*@\2 https://www.percona.com\[email protected];p}}' | while read -r version url; do
distro_list=${distro_list:-}
distro_list=$($download_tool $url | sed -r -n '/Select Software Platform/{[email protected]/@[email protected];[email protected]<\/option>@\[email protected];p}' | sed -r -n '[email protected]*/([^"]*)"[[:space:]]*>.*@\[email protected];/binary|source|select|^$/d;p' | sed -r ':a;N;$!ba;[email protected]\[email protected] @g;')
echo "${version}|${distro_list}" >> "${version_list_info}"
done

# cat "${version_list_info}"
# 5.7|wheezy jessie stretch rhel6 rhel7 precise trusty xenial yakkety zesty
# 5.6|wheezy jessie stretch rhel5 rhel6 rhel7 precise trusty xenial yakkety zesty
# 5.5|wheezy jessie rhel5 rhel6 rhel7 precise trusty xenial yakkety zesty
# 5.1|squeeze rhel5 rhel6 lucid precise


# - distro|version_list
sed -r '[email protected]*\|@@g' "${version_list_info}" | sed -r ':a;N;$!ba;[email protected]\[email protected] @g;[email protected] @\[email protected]' | awk '!a[$0]++' | while read -r distro; do
version_list=$(awk -F\| 'match($NF,/[[:space:]]*rhel6[[:space:]]*/){print $1}' "${version_list_info}" | sed -r ':a;N;$!ba;[email protected]\[email protected] @g;')
echo "$distro|$version_list"
done

# wheezy|5.7 5.6 5.5 5.1
# jessie|5.7 5.6 5.5 5.1
# stretch|5.7 5.6 5.5 5.1
# rhel6|5.7 5.6 5.5 5.1
# rhel7|5.7 5.6 5.5 5.1
# precise|5.7 5.6 5.5 5.1
# trusty|5.7 5.6 5.5 5.1
# xenial|5.7 5.6 5.5 5.1
# yakkety|5.7 5.6 5.5 5.1
# zesty|5.7 5.6 5.5 5.1
# rhel5|5.7 5.6 5.5 5.1
# squeeze|5.7 5.6 5.5 5.1
# lucid|5.7 5.6 5.5 5.1

[[ -f "${version_list_info}" ]] && rm -f "${version_list_info}"
unset version_list_info

Parallel

利用parallel命令並行操作,較節省時間

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
#!/usr/bin/env bash
download_tool=${download_tool:-'curl -fsL'}

version_list_info=$(mktemp -t tempXXXXX.txt)
export version_list_info="${version_list_info}"

# - version|distro_list
funcDistroListForPerVersion(){
# $1 = 5.7 https://www.percona.com/downloads/Percona-Server-LATEST/
local item=${1:-}
if [[ -n "${item}" ]]; then
local version=${version:-}
local version_url=${version_url:-}
local distro_list=${distro_list:-}
version="${item%% *}"
version_url="${item##* }"
distro_list=$($download_tool $version_url | sed -r -n '/Select Software Platform/{[email protected]/@[email protected];[email protected]<\/option>@\[email protected];p}' | sed -r -n '[email protected]*/([^"]*)"[[:space:]]*>.*@\[email protected];/binary|source|select|^$/d;p' | sed -r ':a;N;$!ba;[email protected]\[email protected] @g;')
echo "${version}|${distro_list}" >> "${version_list_info}"
fi
}

export -f funcDistroListForPerVersion
export download_tool="${download_tool}"
$download_tool https://www.percona.com/downloads/ | sed -r -n '/A drop-in replacement for MySQL/,/<\/div>/{/<a/{[email protected]* href="([^"]*)".*>Download ([[:digit:].]+).*@\2 https://www.percona.com\[email protected];p}}' | parallel -k -j 0 funcDistroListForPerVersion 2> /dev/null

# cat "${version_list_info}"
# 5.7|wheezy jessie stretch rhel6 rhel7 precise trusty xenial yakkety zesty
# 5.6|wheezy jessie stretch rhel5 rhel6 rhel7 precise trusty xenial yakkety zesty
# 5.5|wheezy jessie rhel5 rhel6 rhel7 precise trusty xenial yakkety zesty
# 5.1|squeeze rhel5 rhel6 lucid precise

# - distro|version_list
funcVersionListPerDistro(){
local distro=${1:-}
local version_list=${version_list:-}
version_list=$(awk -F\| 'match($NF,/[[:space:]]*'"${distro}"'[[:space:]]*/){print $1}' "${version_list_info}" | sed -r ':a;N;$!ba;[email protected]\[email protected] @g;')
echo "$distro|$version_list"
}

export version_list_info="${version_list_info}"
export -f funcVersionListPerDistro
sed -r '[email protected]*\|@@g' "${version_list_info}" | sed -r ':a;N;$!ba;[email protected]\[email protected] @g;[email protected] @\[email protected]' | awk '!a[$0]++' | parallel -k -j 0 funcVersionListPerDistro 2> /dev/null

# wheezy|5.7 5.6 5.5 5.1
# jessie|5.7 5.6 5.5 5.1
# stretch|5.7 5.6 5.5 5.1
# rhel6|5.7 5.6 5.5 5.1
# rhel7|5.7 5.6 5.5 5.1
# precise|5.7 5.6 5.5 5.1
# trusty|5.7 5.6 5.5 5.1
# xenial|5.7 5.6 5.5 5.1
# yakkety|5.7 5.6 5.5 5.1
# zesty|5.7 5.6 5.5 5.1
# rhel5|5.7 5.6 5.5 5.1
# squeeze|5.7 5.6 5.5 5.1
# lucid|5.7 5.6 5.5 5.1

[[ -f "${version_list_info}" ]] && rm -f "${version_list_info}"
unset version_list_info

Output

將Shell Script代碼段合併後得到的結果如下

Distro Percona Version
wheezy 5.7, 5.6, 5.5, 5.1
jessie 5.7, 5.6, 5.5, 5.1
stretch 5.7, 5.6, 5.5, 5.1
rhel6 5.7, 5.6, 5.5, 5.1
rhel7 5.7, 5.6, 5.5, 5.1
precise 5.7, 5.6, 5.5, 5.1
trusty 5.7, 5.6, 5.5, 5.1
xenial 5.7, 5.6, 5.5, 5.1
yakkety 5.7, 5.6, 5.5, 5.1
zesty 5.7, 5.6, 5.5, 5.1
rhel5 5.7, 5.6, 5.5, 5.1
squeeze 5.7, 5.6, 5.5, 5.1
lucid 5.7, 5.6, 5.5, 5.1

Performance Comparison

使用bash內置命令time對操作進行計時

1
2
3
4
5
6
7
8
9
# Original Time
real 0m4.429s
user 0m0.136s
sys 0m0.024s

# Parallel Time
real 0m2.456s
user 0m0.544s
sys 0m0.160s

可以看到使用parallel,操作耗時節省接近一半時間。

Reference

Change Logs

  • 2017.08.13 21:26 Sun Asia/Shanghai
    • 初稿完成