1. Cron-like Scheduler
1.1 Python任務調度框架 APScheduler
一個基於Python,提供類似Cron功能,並深受Java Quartz 影響的輕量級進程內任務調度框架。
Advanced Python Scheduler (APScheduler) is a light but powerful in-process task scheduler that lets you schedule jobs (functions or any python callables) to be executed at times of your choosing.
This can be a far better alternative to externally run cron scripts for long-running applications (e.g. web applications), as it is platform neutral and can directly access your application's variables and functions.
The development of APScheduler was heavily influenced by theQuartz task scheduler written in Java. APScheduler provides most of the major features that Quartz does, but it also provides features not present in Quartz (such as multiple job stores).
框架工具資源:
1.2 任務調度框架 cron4j
cron4j 是一個Java的任務調度框架,類似於UNIX系統下的crontab.
使用示例:
1.3 conclusion:
not web-based applicationneed to programjust only scheduler2. Gearman分布式遠程過程處理框架
2.1Outline
Gearmand 是 Gearman 的作業伺服器組件,Gearman是一個分發任務的程序框架,可以用在各種場合,與Hadoop相 比,Gearman更偏向於任務分發功能。它的 任務分布非常 簡單,簡單得可以只需要用腳本即可完成。Gearman最初用於LiveJournal的圖片resize功能,由於圖片resize需要消耗大量計算資 源,因此需要調度到後端多臺伺服器執行,完成任務之後返回前端再呈現到界面。
2.2 Features
Open Source - It's free! (in both meanings of the word) Gearman has an active open source community that is easy to get involved with if you need help or want to contribute.
Multi-language - There are interfaces for a number of languages, and this list is growing. You also have the option to write heterogeneous applications with clients submitting work in one language and workers performing that work in another.
Flexible - You are not tied to any specific design pattern. You can quickly put together distributed applications using any model you choose, one of those options being Map/Reduce.
Fast - Gearman has a simple protocol and interface with a new optimized server in C to minimize your application overhead.
Embeddable - Since Gearman is fast and lightweight, it is great for applications of all sizes. It is also easy to introduce into existing applications with minimal overhead.
No single point of failure - Gearman can not only help scale systems, but can do it in a fault tolerant way.
3 ETL 商業免費工具: TASKCTL Web應用版
TASKCTL免費Web版作為目前唯一的ETL調度領域商業級免費軟體,保證100% free,絕無黑盒代碼。它志在促進該領域的獨立發展,使調度在ETL領域獨立化、專業化、系統化。從而使項目實施更輕鬆便捷,使企業基礎架構更清晰、更易管理。
(一)主要適用環境
作業系統:aix/linux/unix等(由於採用標準c語言構建,理論上可應用於各種主流unix系列)
項目規模:適用於中小型ETL項目
ETL工具環境:TASKCTL由於採用任務插件驅動機制,因此,可支持各種存儲過程、各種腳本、以及諸如Datastage\Informatica\kettle等各種ETL工具任務。
(二)主要功能
核心調度功能:主要可以完成串行、並行、依賴、互斥、執行計劃、定時、容錯、循環、條件分支、遠程、負載均衡、自定義條件等各種不同的核心調度功能。
擴展功能
網絡擴展:可實現單機部署、多服務部署、遠程代理部署、集群部署等多種網絡部署
應用擴展:技術平臺設計有專門的應用API接口,可實現更多的調度應用。
任務類型擴展:為了適應不同類型的任務調度,平臺可通過具有統一模版、統一接口的插件進行快速擴展。
應用功能:配置功能、流程設計功能、監控功能、各種查詢功能以及諸如重跑、重置等人工幹預功能。
(三) 主要創新
無資料庫設計:國內首款專業無資料庫調度技術平臺。
插件機制:業界唯一通過具有統一應用接口的插件來擴展任務類型的技術平臺。
流程設計代碼開發設計理念:調度領域唯一通過文本代碼設計流程的調度技術平臺。具有語法代碼特徵的文本代碼設計與傳統記錄表格對話框方式相比,操作更方便、設計更靈活、可讀性更強。