上面的場景,引出了本文主題,用戶升級系統之後,如何能通過命令來修復設備信息中舊設備信息的設備高度。
按要求創建命令目錄結構
給 django 應用加命令其實很簡單,只需要在應用程式中添加一個 management/commands 目錄。Django 會給目錄下的每個 Python 模塊註冊一個 manage.py 命令,這個命令的名字不以下劃線開頭。例如:
idcops/
__init__.py
models.py
management/
__init__.py
commands/
__init__.py
_private.py
fixdevicedata.py
tests.py
views.py
_private.py 模塊將不會作為管理命令使用。fixdevicedata.py 必須定義 Command 類,該類繼承自 BaseCommand 或其 子類。編寫自定義命令
以下是 fixdevicedata.py 自定義 django-admin 命令的源碼:
from queue import Queue
from threading import Thread
from django.core.management.base import BaseCommand, CommandError
from django.core.paginator import Paginator
from idcops.models import Device
class Command(BaseCommand):
help = "更新所有設備的高度(U)、U位範圍"
batch_size = 256
def add_arguments(self, parser):
parser.add_argument(
'--size', type=int, action='store',
dest='size', default=self.batch_size,
help=f"指定每個線程處理多少條設備信息,默認是: {self.batch_size}/thread",
)
def fix_device_height(self, obj):
# 主要業務邏輯
height = obj.units.all().count()
Device.objects.filter(pk=obj.pk).update(height=height)
Device.objects.filter(pk=obj.pk).update(urange=obj.list_units())
def handle(self, *args, **options):
per_page = options['size']
objects = Device.objects.filter()
page = Paginator(objects, per_page=per_page)
object_list = page.object_list
try:
if object_list.exists():
q = Queue()
def worker():
while True:
if q.empty():
return
else:
item = q.get()
self.fix_device_height(item)
q.task_done()
for obj in object_list:
q.put(obj)
threads = list()
[
threads.append(Thread(target=worker, daemon=True))
for _ in range(page.num_pages)
]
[t.start() for t in threads]
[t.join() for t in threads]
print(
f"threads number: {len(threads)} ({per_page}/thread), total: {page.count}"
)
else:
print('No objects')
except CommandError as e:
raise(e)
add_arguments 給命令添加一個可選參數 size巧妙利用 Django 分頁,配合 queue 多線程處理業務邏輯
自定義命令效果
(env) PS D:\Personal\django-idcops> python .\manage.py fixdevicedata -h
usage: manage.py fixdevicedata [-h] [--size SIZE] [--version] [-v {0,1,2,3}]
[--settings SETTINGS] [--pythonpath PYTHONPATH]
[--traceback] [--no-color]
更新所有設備的高度(U)、U位範圍
optional arguments:
-h, --help show this help message and exit
--size SIZE 指定每個線程處理多少條設備信息,默認是: 256/thread
--version show program's version number and exit
-v {0,1,2,3}, --verbosity {0,1,2,3}
Verbosity level; 0=minimal output, 1=normal output,
2=verbose output, 3=very verbose output
--settings SETTINGS The Python path to a settings module, e.g.
"myproject.settings.main". If this isn't provided, the
DJANGO_SETTINGS_MODULE environment variable will be
used.
--pythonpath PYTHONPATH
A directory to add to the Python path, e.g.
"/home/djangoprojects/myproject".
--traceback Raise on CommandError exceptions
--no-color Don't colorize the command output.
(env) PS D:\Personal\django-idcops>(env) PS D:\Personal\django-idcops> python .\manage.py fixdevicedata --size 2
threads number: 4 (2/thread), total: 7
(env) PS D:\Personal\django-idcops> python .\manage.py fixdevicedata
threads number: 1 (256/thread), total: 7
一共7條數據,共啟動了4個線程,每個線程能處理2條數據一共7條數據,共啟動了1個線程,每個線程能處理256條數據
總結:
Django 會給目錄下的每個 Python 模塊註冊一個 manage.py 命令。所以,一個模塊只做一類命令的業務。批量業務數據量比較大,可以使用多線程來加速業務的處理。