Amazon Redshift - 資料庫開發人員指南...Amazon Redshift 資料庫開發人員指南...

Amazon Redshift資料庫開發人員指南

Amazon Redshift 資料庫開發人員指南

Amazon Redshift: 資料庫開發人員指南Copyright © 2020 Amazon Web Services, Inc. and/or its affiliates. All rights reserved.

Amazon's trademarks and trade dress may not be used in connection with any product or service that is not Amazon's,in any manner that is likely to cause confusion among customers, or in any manner that disparages or discreditsAmazon. All other trademarks not owned by Amazon are the property of their respective owners, who may or may notbe affiliated with, connected to, or sponsored by Amazon.


Table of Contents系統概觀 ............................................................................................................................................ 1

您是第一次使用 Amazon Redshift 的新手嗎？ ................................................................................. 1您是資料庫開發人員嗎？ .............................................................................................................. 1先決條件 .................................................................................................................................... 2系統和架構概觀 .......................................................................................................................... 3

資料倉儲系統架構 ............................................................................................................... 3效能 .................................................................................................................................. 5直欄式儲存 ........................................................................................................................ 7內部架構與系統操作 ............................................................................................................ 8工作負載管理 ..................................................................................................................... 8搭配其他服務使用 Amazon Redshift ....................................................................................... 9

開始使用資料庫 ................................................................................................................................. 11步驟 1：建立資料庫 ................................................................................................................... 11步驟 2：建立資料庫使用者 ......................................................................................................... 11

刪除資料庫使用者 .............................................................................................................. 12步驟 3：建立資料庫資料表 ......................................................................................................... 12

將資料列插入至資料表 ....................................................................................................... 13從資料表選取資料 .............................................................................................................. 13

步驟 4：載入範例資料 ................................................................................................................ 13步驟 5：查詢系統資料表 ............................................................................................................. 13

檢視資料表名稱的清單 ....................................................................................................... 14檢視資料庫使用者 .............................................................................................................. 14檢視近期查詢 .................................................................................................................... 15判斷執行中查詢的程序 ID ................................................................................................... 15

步驟 6：取消查詢 ...................................................................................................................... 16取消另一個工作階段的查詢 ................................................................................................. 17使用超級使用者佇列來取消查詢 ........................................................................................... 17

步驟 7：清理您的資源 ................................................................................................................ 17最佳實務 .......................................................................................................................................... 19

設計資料表的最佳實務 ............................................................................................................... 19參加調校資料表設計教學 .................................................................................................... 19選擇最佳的排序索引鍵 ....................................................................................................... 20選擇最佳的分佈方式 .......................................................................................................... 20使用自動壓縮 .................................................................................................................... 20定義限制條件 .................................................................................................................... 21使用可能的最小欄位大小 .................................................................................................... 21在日期欄位中使用日期/時間資料類型 .................................................................................... 21

載入資料最佳實務 ...................................................................................................................... 21參加載入資料教學 .............................................................................................................. 22參加調校資料表設計教學 .................................................................................................... 22使用 COPY 命令載入資料 ................................................................................................... 22使用單一 COPY 命令 ......................................................................................................... 22將您的載入資料分割為多個檔案 ........................................................................................... 22壓縮您的資料檔案 .............................................................................................................. 22使用資訊清單檔案 .............................................................................................................. 23在載入前後驗證資料檔案 .................................................................................................... 23使用多資料列插入 .............................................................................................................. 23使用大量插入 .................................................................................................................... 23以排序索引鍵順序載入資料 ................................................................................................. 24將資料載入循序區塊 .......................................................................................................... 24使用時間序列資料表 .......................................................................................................... 24使用預備資料表執行合併 .................................................................................................... 24排程維護時段 .................................................................................................................... 24

設計查詢的最佳實務 .................................................................................................................. 25

iii


使用 Advisor ............................................................................................................................. 26Amazon Redshift 區域 ....................................................................................................... 26存取 Advisor ..................................................................................................................... 26Advisor 建議 ..................................................................................................................... 28

教學課程 .......................................................................................................................................... 38設計資料表 ....................................................................................................................................... 39

選擇資料欄壓縮類型 .................................................................................................................. 39壓縮編碼 .......................................................................................................................... 40測試壓縮編碼 .................................................................................................................... 46範例：選擇 CUSTOMER 資料表的壓縮編碼 ........................................................................... 47

選擇資料分佈樣式 ...................................................................................................................... 49資料分佈概念 .................................................................................................................... 49分佈樣式 .......................................................................................................................... 50檢視分佈樣式 .................................................................................................................... 51評估查詢模式 .................................................................................................................... 51指定分佈樣式 .................................................................................................................... 52評估查詢計畫 .................................................................................................................... 52查詢計劃範例 .................................................................................................................... 54分佈範例 .......................................................................................................................... 58

選擇排序索引鍵 ......................................................................................................................... 59複合排序索引鍵 ................................................................................................................. 60交錯排序索引鍵 ................................................................................................................. 60比較排序樣式 .................................................................................................................... 61

定義限制 .................................................................................................................................. 64分析資料表設計 ......................................................................................................................... 64教學：調校資料表設計 ............................................................................................................... 66

先決條件 .......................................................................................................................... 66步驟 ................................................................................................................................ 66步驟 1：建立測試資料集 ..................................................................................................... 66步驟 2：建立基準 .............................................................................................................. 70步驟 3：選取排序索引鍵 ..................................................................................................... 73步驟 4：選取分佈樣式 ........................................................................................................ 74步驟 5：檢閱壓縮編碼 ........................................................................................................ 77步驟 6：重新建立測試資料集 .............................................................................................. 79步驟 7：調校之後重新測試系統效能 ..................................................................................... 82步驟 8：評估結果 .............................................................................................................. 86步驟 9：清理您的資源 ........................................................................................................ 87總結 ................................................................................................................................ 88

載入資料 .......................................................................................................................................... 89使用 COPY 載入資料 ................................................................................................................. 89

登入資料和存取許可 .......................................................................................................... 90準備您的輸入資料 .............................................................................................................. 91從 Amazon S3 載入資料 .................................................................................................... 91從 Amazon EMR 載入資料 ................................................................................................. 99從遠端主機載入資料 ......................................................................................................... 103從 Amazon DynamoDB 載入 ............................................................................................. 109驗證資料已正確載入 ......................................................................................................... 111驗證輸入資料 .................................................................................................................. 111自動壓縮 ........................................................................................................................ 111針對縮小資料表進行優化 ................................................................................................... 113預設值 ........................................................................................................................... 113疑難排解 ........................................................................................................................ 114

使用 DML 更新 ....................................................................................................................... 118更新和插入 ............................................................................................................................. 118

合併方法 1：取代現有資料列 ............................................................................................. 119合併方法 2：指定欄位清單 ................................................................................................ 119建立暫時的預備資料表 ...................................................................................................... 119

iv


取代現有資料列來執行合併操作 ......................................................................................... 119指定欄位清單來執行合併操作 ............................................................................................. 120合併範例 ........................................................................................................................ 121

執行深層複製 .......................................................................................................................... 123分析資料表 ............................................................................................................................. 125

自動分析 ........................................................................................................................ 125新資料表資料的分析 ......................................................................................................... 125ANALYZE 命令歷史記錄 ................................................................................................... 128

清空資料表 ............................................................................................................................. 129自動資料表排序 ............................................................................................................... 129自動清空刪除 .................................................................................................................. 130VACUUM 頻率 ................................................................................................................ 130排序階段和合併階段 ......................................................................................................... 130清空閾值 ........................................................................................................................ 131清空類型 ........................................................................................................................ 131管理清空時間 .................................................................................................................. 131

管理並行寫入操作 .................................................................................................................... 137可序列化隔離 .................................................................................................................. 137寫入和讀寫操作 ............................................................................................................... 139並行寫入範例 .................................................................................................................. 140

教學：從 Amazon S3 載入資料 ................................................................................................. 141先決條件 ........................................................................................................................ 141概觀 .............................................................................................................................. 142步驟 .............................................................................................................................. 142步驟 1：建立叢集 ............................................................................................................ 142步驟 2：下載資料檔案 ...................................................................................................... 144步驟 3：將檔案上傳至 Amazon S3 儲存貯體 ....................................................................... 144步驟 4：建立範例資料表 ................................................................................................... 145步驟 5：執行 COPY 命令 ................................................................................................. 148步驟 6：清空及分析資料庫 ................................................................................................ 159步驟 7：清理您的資源 ...................................................................................................... 159總結 .............................................................................................................................. 160

卸載資料 ........................................................................................................................................ 161將資料卸載至 Amazon S3 ......................................................................................................... 161卸載加密的資料檔案 ................................................................................................................. 164以分隔或固定寬度的格式卸載資料 .............................................................................................. 165重新載入已卸載資料 ................................................................................................................. 166

建立使用者定義的函數 ...................................................................................................................... 167UDF 安全與權限 ...................................................................................................................... 167建立純量 SQL UDF ................................................................................................................. 167

純量 SQL 函數範例 ......................................................................................................... 168建立純量 Python UDF .............................................................................................................. 168

純量 Python UDF 範例 ..................................................................................................... 169Python UDF 資料類型 ...................................................................................................... 169ANYELEMENT 資料類型 .................................................................................................. 170Python 語言支援 ............................................................................................................. 170UDF 限制條件 ................................................................................................................. 173

命名 UDF ............................................................................................................................... 173多載函數名稱 .................................................................................................................. 173防止 UDF 命名衝突 ......................................................................................................... 173

記錄錯誤和警告 ....................................................................................................................... 174建立預存程序 .................................................................................................................................. 175

預存程序概觀 .......................................................................................................................... 175命名預存程序 .................................................................................................................. 177安全和權限 ..................................................................................................................... 177傳回結果集 ..................................................................................................................... 178管理交易 ........................................................................................................................ 179

v


捕捉錯誤 ........................................................................................................................ 184記錄預存程序日誌 ............................................................................................................ 185限制和差異 ..................................................................................................................... 185

PL/pgSQL 語言參考 ................................................................................................................. 185PL/pgSQL 參考慣例 ......................................................................................................... 186PL/pgSQL 的結構 ............................................................................................................ 186支援的 PL/pgSQL 陳述式 .................................................................................................. 190

建立具體化檢視 ............................................................................................................................... 200查詢具體化檢視 ....................................................................................................................... 200

查詢空間資料 .................................................................................................................................. 202限制 ...................................................................................................................................... 202

使用聯合查詢來查詢資料 ................................................................................................................... 204聯合查詢入門 .......................................................................................................................... 204建立秘密和 IAM 角色 ............................................................................................................... 205

先決條件 ........................................................................................................................ 205使用聯合查詢的範例 ................................................................................................................. 206資料類型差異 .......................................................................................................................... 208限制及考量 ............................................................................................................................. 209

使用 Amazon Redshift Spectrum 查詢外部資料 .................................................................................... 211Amazon Redshift Spectrum 概觀 ............................................................................................... 211

Amazon Redshift Spectrum 區域 ....................................................................................... 212Amazon Redshift Spectrum 考量 ....................................................................................... 212

Amazon Redshift Spectrum 入門 ............................................................................................... 213先決條件 ........................................................................................................................ 213步驟 .............................................................................................................................. 213步驟 1.建立 IAM 角色 ...................................................................................................... 213步驟 2：建立 IAM 角色與叢集的關聯 .................................................................................. 216步驟 3：建立外部結構描述與外部資料表 ............................................................................. 216步驟 4：在 Amazon S3 中查詢您的資料 .............................................................................. 217

Amazon Redshift Spectrum 的 IAM 政策 ..................................................................................... 219Amazon S3 許可 ............................................................................................................. 220跨帳戶 Amazon S3 許可 ................................................................................................... 220使用 Redshift Spectrum 授予或限制存取 ............................................................................. 221最低許可 ........................................................................................................................ 221鏈結 IAM 角色 ................................................................................................................ 222存取 AWS Glue 資料 ....................................................................................................... 223

搭配使用 Redshift Spectrum 與 Lake Formation ........................................................................... 228在 Amazon Redshift Spectrum 為查詢建立資料檔案 ...................................................................... 229

Redshift Spectrum 的資料格式 .......................................................................................... 229Redshift Spectrum 的壓縮類型 .......................................................................................... 230Redshift Spectrum 的加密 ................................................................................................ 231

建立外部結構描述 .................................................................................................................... 231使用外部目錄 .................................................................................................................. 232

建立外部資料表 ....................................................................................................................... 237虛擬資料欄 ..................................................................................................................... 238分割 Redshift Spectrum 外部資料表 ................................................................................... 239映射到 ORC 資料欄 ......................................................................................................... 242

改善 Amazon Redshift Spectrum 查詢效能 .................................................................................. 244監控指標 ................................................................................................................................ 246對查詢進行故障診斷 ................................................................................................................. 247

超過重試次數 .................................................................................................................. 247存取已調節 ..................................................................................................................... 248超過資源限制 .................................................................................................................. 248分割的資料表沒有傳回資料列 ............................................................................................. 249未授權的錯誤 .................................................................................................................. 249不相容的資料格式 ............................................................................................................ 249在 Amazon Redshift 使用 Hive DDL 時的語法錯誤 ................................................................ 249

vi


建立暫存資料表的許可 ...................................................................................................... 250教學：使用 Amazon Redshift Spectrum 查詢巢狀資料 ................................................................... 250

概觀 .............................................................................................................................. 250步驟 1：建立包含巢狀資料的外部資料表 ............................................................................. 251步驟 2：在 Amazon S3 中使用 SQL 延伸模組查詢您的巢狀資料 ............................................. 251巢狀資料使用案例 ............................................................................................................ 255巢狀資料限制 .................................................................................................................. 256

調校查詢效能 .................................................................................................................................. 258查詢處理 ................................................................................................................................ 258

查詢計劃和執行工作流程 ................................................................................................... 258查詢計劃 ........................................................................................................................ 260檢閱查詢計劃步驟 ............................................................................................................ 265影響查詢效能的因素 ......................................................................................................... 267

分析和改善查詢 ....................................................................................................................... 267查詢分析工作流程 ............................................................................................................ 268檢閱查詢提醒 .................................................................................................................. 268分析查詢計劃 .................................................................................................................. 269分析查詢摘要 .................................................................................................................. 270改善查詢效能 .................................................................................................................. 274診斷查詢以進行查詢調校 ................................................................................................... 277

對查詢進行故障診斷 ................................................................................................................. 279連線失敗 ........................................................................................................................ 280查詢懸置 ........................................................................................................................ 280查詢耗費太長時間 ............................................................................................................ 281載入失敗 ........................................................................................................................ 281載入耗費太長時間 ............................................................................................................ 282載入資料不正確 ............................................................................................................... 282設定 JDBC 擷取大小參數 .................................................................................................. 282

實作工作負載管理 ............................................................................................................................ 284修改 WLM 組態 ....................................................................................................................... 285

從手動 WLM 遷移至自動 WLM .......................................................................................... 285自動 WLM .............................................................................................................................. 286

優先順序 ........................................................................................................................ 286並行擴展模式 .................................................................................................................. 287使用者群組 ..................................................................................................................... 287查詢群組 ........................................................................................................................ 287萬用字元 ........................................................................................................................ 287查詢監控規則 .................................................................................................................. 287檢查自動 WLM ................................................................................................................ 287查詢優先順序 .................................................................................................................. 288

手動 WLM .............................................................................................................................. 291並行擴展模式 .................................................................................................................. 291並行層級 ........................................................................................................................ 292使用者群組 ..................................................................................................................... 293查詢群組 ........................................................................................................................ 293萬用字元 ........................................................................................................................ 293要使用的 WLM 記憶體百分比 ............................................................................................ 293WLM 逾時 ...................................................................................................................... 293查詢監控規則 .................................................................................................................. 294WLM 查詢佇列跳轉 .......................................................................................................... 294教學：設定手動 WLM 佇列 ............................................................................................... 296

並行擴展 ................................................................................................................................ 309並行擴展區域 .................................................................................................................. 309並行擴展候選項目 ............................................................................................................ 310設定並行擴展佇列 ............................................................................................................ 288監控並行擴展 .................................................................................................................. 310系統檢視 ........................................................................................................................ 311

vii


短期查詢加速 .......................................................................................................................... 311SQA 最長執行時間 .......................................................................................................... 311監控 SQA ....................................................................................................................... 312

WLM 佇列指派規則 .................................................................................................................. 312佇列指派範例 .................................................................................................................. 313

將查詢指派給佇列 .................................................................................................................... 315根據使用者群組將查詢指派給佇列 ...................................................................................... 315將查詢指派給查詢群組 ...................................................................................................... 315將查詢指派給超級使用者佇列 ............................................................................................. 315

動態和靜態屬性 ....................................................................................................................... 316WLM 動態記憶體配置 ....................................................................................................... 316動態 WLM 範例 ............................................................................................................... 317

查詢監控規則 .......................................................................................................................... 318定義查詢監控規則 ............................................................................................................ 318查詢監控指標 .................................................................................................................. 320查詢監控規則範本 ............................................................................................................ 321查詢監控規則的系統資料表和檢視 ...................................................................................... 322

WLM 系統資料表和檢視 ............................................................................................................ 322WLM 服務類別 ID ............................................................................................................ 323

管理資料庫安全 ............................................................................................................................... 325Amazon Redshift 安全概觀 ....................................................................................................... 325預設資料庫使用者權限 .............................................................................................................. 326超級使用者 ............................................................................................................................. 326使用者 ................................................................................................................................... 326

建立、更改和刪除使用者 ................................................................................................... 327群組 ...................................................................................................................................... 327

建立、更改和刪除群組 ...................................................................................................... 327結構描述 ................................................................................................................................ 328

建立、更改和刪除結構描述 ................................................................................................ 328搜尋路徑 ........................................................................................................................ 328結構描述型權限 ............................................................................................................... 329

控制使用者和群組存取的範例 ..................................................................................................... 329SQL 參考 ....................................................................................................................................... 331

Amazon Redshift SQL ............................................................................................................. 331領導節點上所支援的 SQL 函數 .......................................................................................... 331Amazon Redshift 與 PostgreSQL ....................................................................................... 332

使用 SQL ............................................................................................................................... 337SQL 參考慣例 ................................................................................................................. 337基本元素 ........................................................................................................................ 338表達式 ........................................................................................................................... 361條件 .............................................................................................................................. 364

SQL 命令 ............................................................................................................................... 381ABORT .......................................................................................................................... 382ALTER DATABASE ......................................................................................................... 383ALTER DEFAULT PRIVILEGES ........................................................................................ 385ALTER GROUP .............................................................................................................. 387ALTER PROCEDURE ...................................................................................................... 388ALTER SCHEMA ............................................................................................................ 389ALTER TABLE ................................................................................................................ 390ALTER TABLE APPEND .................................................................................................. 400ALTER USER ................................................................................................................. 404ANALYZE ...................................................................................................................... 407ANALYZE COMPRESSION .............................................................................................. 409BEGIN ........................................................................................................................... 410CALL ............................................................................................................................. 411CANCEL ........................................................................................................................ 414CLOSE .......................................................................................................................... 415

viii


COMMENT ..................................................................................................................... 416COMMIT ........................................................................................................................ 417COPY ............................................................................................................................ 418CREATE DATABASE ...................................................................................................... 472CREATE EXTERNAL SCHEMA ........................................................................................ 474CREATE EXTERNAL TABLE ............................................................................................ 478CREATE FUNCTION ....................................................................................................... 489CREATE GROUP ............................................................................................................ 493CREATE LIBRARY .......................................................................................................... 494CREATE MATERIALIZED VIEW ........................................................................................ 496CREATE PROCEDURE ................................................................................................... 498CREATE SCHEMA .......................................................................................................... 500CREATE TABLE ............................................................................................................. 502CREATE TABLE AS ........................................................................................................ 514CREATE USER .............................................................................................................. 522CREATE VIEW ............................................................................................................... 525DEALLOCATE ................................................................................................................ 527DECLARE ...................................................................................................................... 528DELETE ........................................................................................................................ 530DROP DATABASE .......................................................................................................... 532DROP FUNCTION ........................................................................................................... 532DROP GROUP ............................................................................................................... 533DROP LIBRARY ............................................................................................................. 534DROP MATERIALIZED VIEW ........................................................................................... 534DROP PROCEDURE ....................................................................................................... 535DROP SCHEMA ............................................................................................................. 535DROP TABLE ................................................................................................................. 537DROP USER .................................................................................................................. 539DROP VIEW .................................................................................................................. 540結束 .............................................................................................................................. 541EXECUTE ...................................................................................................................... 542EXPLAIN ....................................................................................................................... 543FETCH .......................................................................................................................... 546GRANT .......................................................................................................................... 548INSERT ......................................................................................................................... 554LOCK ............................................................................................................................ 558PREPARE ...................................................................................................................... 559REFRESH MATERIALIZED VIEW ..................................................................................... 560RESET .......................................................................................................................... 562REVOKE ........................................................................................................................ 563ROLLBACK .................................................................................................................... 568SELECT ........................................................................................................................ 569SELECT INTO ................................................................................................................ 595SET .............................................................................................................................. 596SET SESSION AUTHORIZATION ...................................................................................... 599SET SESSION CHARACTERISTICS .................................................................................. 600SHOW ........................................................................................................................... 600SHOW PROCEDURE ...................................................................................................... 600START TRANSACTION ................................................................................................... 601TRUNCATE .................................................................................................................... 601UNLOAD ........................................................................................................................ 602更新 .............................................................................................................................. 618VACUUM ....................................................................................................................... 623

SQL 函數參考 ......................................................................................................................... 626僅限於領導節點的函數 ...................................................................................................... 627僅限於運算節點的函數 ...................................................................................................... 628彙總函數 ........................................................................................................................ 628

ix


位元彙整函數 .................................................................................................................. 643視窗函數 ........................................................................................................................ 647條件表達式 ..................................................................................................................... 690日期和時間函數 ............................................................................................................... 698空間函數 ........................................................................................................................ 733數學函數 ........................................................................................................................ 776字串函數 ........................................................................................................................ 799雜湊函數 ........................................................................................................................ 836JSON 函數 ..................................................................................................................... 838資料類型格式化函數 ......................................................................................................... 844系統管理函數 .................................................................................................................. 853系統資訊函數 .................................................................................................................. 860

保留字 ................................................................................................................................... 873系統資料表參考 ............................................................................................................................... 877

系統資料表和檢視 .................................................................................................................... 877系統資料表和檢視的類型 ........................................................................................................... 877系統資料表和檢視中資料的可見性 .............................................................................................. 878

篩選系統產生的查詢 ......................................................................................................... 878用於記錄日誌的 STL 資料表 ...................................................................................................... 878

STL_AGGR .................................................................................................................... 880STL_ALERT_EVENT_LOG ................................................................................................ 882STL_ANALYZE ............................................................................................................... 883STL_ANALYZE_COMPRESSION ....................................................................................... 884STL_BCAST ................................................................................................................... 886STL_COMMIT_STATS ..................................................................................................... 887STL_CONNECTION_LOG ................................................................................................. 888STL_DDLTEXT ............................................................................................................... 889STL_DELETE ................................................................................................................. 892STL_DISK_FULL_DIAG .................................................................................................... 894STL_DIST ...................................................................................................................... 895STL_ERROR .................................................................................................................. 896STL_EXPLAIN ................................................................................................................ 897STL_FILE_SCAN ............................................................................................................. 899STL_HASH ..................................................................................................................... 900STL_HASHJOIN .............................................................................................................. 901STL_INSERT .................................................................................................................. 903STL_LIMIT ..................................................................................................................... 903STL_LOAD_COMMITS ..................................................................................................... 905STL_LOAD_ERRORS ...................................................................................................... 907STL_LOADERROR_DETAIL .............................................................................................. 909STL_MERGE .................................................................................................................. 910STL_MERGEJOIN ........................................................................................................... 911STL_MV_STATE ............................................................................................................. 912STL_NESTLOOP ............................................................................................................. 914STL_PARSE ................................................................................................................... 915STL_PLAN_INFO ............................................................................................................ 916STL_PROJECT ............................................................................................................... 918STL_QUERY .................................................................................................................. 919STL_QUERY_METRICS ................................................................................................... 921STL_QUERYTEXT ........................................................................................................... 924STL_REPLACEMENTS .................................................................................................... 926STL_RESTARTED_SESSIONS ......................................................................................... 927STL_RETURN ................................................................................................................ 927STL_S3CLIENT ............................................................................................................... 928STL_S3CLIENT_ERROR .................................................................................................. 930STL_SAVE ..................................................................................................................... 931STL_SCAN ..................................................................................................................... 932

x


STL_SCHEMA_QUOTA_VIOLATIONS ................................................................................ 934STL_SESSIONS .............................................................................................................. 935STL_SORT ..................................................................................................................... 936STL_SSHCLIENT_ERROR ............................................................................................... 938STL_STREAM_SEGS ...................................................................................................... 938STL_TR_CONFLICT ........................................................................................................ 939STL_UNDONE ................................................................................................................ 940STL_UNIQUE ................................................................................................................. 940STL_UNLOAD_LOG ........................................................................................................ 942STL_USAGE_CONTROL .................................................................................................. 943STL_USERLOG .............................................................................................................. 943STL_UTILITYTEXT .......................................................................................................... 945STL_VACUUM ................................................................................................................ 947STL_WINDOW ................................................................................................................ 949STL_WLM_ERROR ......................................................................................................... 950STL_WLM_RULE_ACTION ............................................................................................... 950STL_WLM_QUERY .......................................................................................................... 951

快照資料的 STV 資料表 ............................................................................................................ 953STV_ACTIVE_CURSORS ................................................................................................. 954STV_BLOCKLIST ............................................................................................................ 954STV_CURSOR_CONFIGURATION .................................................................................... 957STV_EXEC_STATE ......................................................................................................... 957STV_INFLIGHT ............................................................................................................... 958STV_LOAD_STATE ......................................................................................................... 959STV_LOCKS ................................................................................................................... 960STV_MV_INFO ............................................................................................................... 961STV_PARTITIONS ........................................................................................................... 962STV_QUERY_METRICS ................................................................................................... 964STV_RECENTS .............................................................................................................. 967STV_SESSIONS ............................................................................................................. 968STV_SLICES .................................................................................................................. 969STV_STARTUP_RECOVERY_STATE ................................................................................ 969STV_TBL_PERM ............................................................................................................. 970STV_TBL_TRANS ........................................................................................................... 972STV_WLM_QMR_CONFIG ................................................................................................ 973STV_WLM_CLASSIFICATION_CONFIG .............................................................................. 974STV_WLM_QUERY_QUEUE_STATE ................................................................................. 975STV_WLM_QUERY_STATE .............................................................................................. 976STV_WLM_QUERY_TASK_STATE .................................................................................... 977STV_WLM_SERVICE_CLASS_CONFIG .............................................................................. 978STV_WLM_SERVICE_CLASS_STATE ................................................................................ 979

系統檢視 ................................................................................................................................ 980SVCS 檢視 ..................................................................................................................... 981SVL 檢視 ....................................................................................................................... 997SVV 檢視 ..................................................................................................................... 1025

系統目錄資料表 ..................................................................................................................... 1042PG_ATTRIBUTE_INFO ................................................................................................... 1042PG_CLASS_INFO .......................................................................................................... 1042PG_DATABASE_INFO .................................................................................................... 1044PG_DEFAULT_ACL ....................................................................................................... 1044PG_EXTERNAL_SCHEMA .............................................................................................. 1045PG_LIBRARY ................................................................................................................ 1046PG_PROC_INFO ........................................................................................................... 1047PG_STATISTIC_INDICATOR ........................................................................................... 1047PG_TABLE_DEF ........................................................................................................... 1048查詢目錄資料表 ............................................................................................................. 1050

組態參考 ...................................................................................................................................... 1054

xi


修改伺服器組態 ..................................................................................................................... 1054analyze_threshold_percent ...................................................................................................... 1055

值 (粗體為預設值) .......................................................................................................... 1055描述 ............................................................................................................................. 1055範例 ............................................................................................................................. 1055

auto_analyze ......................................................................................................................... 1055值 (粗體為預設值) .......................................................................................................... 1055描述 ............................................................................................................................. 1055範例 ............................................................................................................................. 1055

datestyle .............................................................................................................................. 1056值 (粗體為預設值) .......................................................................................................... 1056描述 ............................................................................................................................. 1056範例 ............................................................................................................................. 1056

describe_field_name_in_uppercase ........................................................................................... 1056值 (粗體為預設值) .......................................................................................................... 1056描述 ............................................................................................................................. 1056範例 ............................................................................................................................. 1056

enable_result_cache_for_session .............................................................................................. 1056值 (粗體為預設值) .......................................................................................................... 1056描述 ............................................................................................................................. 1056

enable_vacuum_boost ............................................................................................................ 1057值 (粗體為預設值) .......................................................................................................... 1057描述 ............................................................................................................................. 1056

extra_float_digits .................................................................................................................... 1057值 (粗體為預設值) .......................................................................................................... 1057描述 ............................................................................................................................. 1057

max_concurrency_scaling_clusters ........................................................................................... 1057值 (粗體為預設值) .......................................................................................................... 1057描述 ............................................................................................................................. 1057

max_cursor_result_set_size ..................................................................................................... 1058值 (粗體為預設值) .......................................................................................................... 1058描述 ............................................................................................................................. 1058

query_group .......................................................................................................................... 1058值 (粗體為預設值) .......................................................................................................... 1058描述 ............................................................................................................................. 1058

search_path .......................................................................................................................... 1058值 (粗體為預設值) .......................................................................................................... 1058描述 ............................................................................................................................. 1059範例 ............................................................................................................................. 1059

statement_timeout .................................................................................................................. 1060值 (粗體為預設值) .......................................................................................................... 1060描述 ............................................................................................................................. 1060範例 ............................................................................................................................. 1060

stored_proc_log_min_messages ............................................................................................... 1060值 (粗體為預設值) .......................................................................................................... 1060描述 ............................................................................................................................. 1056

timezone .............................................................................................................................. 1061值 (粗體為預設值) .......................................................................................................... 1061語法 ............................................................................................................................. 1061描述 ............................................................................................................................. 1061時區格式 ...................................................................................................................... 1061範例 ............................................................................................................................. 1062

wlm_query_slot_count ............................................................................................................. 1063值 (粗體為預設值) .......................................................................................................... 1063描述 ............................................................................................................................. 1063範例 ............................................................................................................................. 1063

範本資料庫 ................................................................................................................................... 1065

xii


CATEGORY 資料表 ............................................................................................................... 1066DATE 資料表 ........................................................................................................................ 1066EVENT 資料表 ...................................................................................................................... 1067VENUE 資料表 ...................................................................................................................... 1067USERS 資料表 ...................................................................................................................... 1067LISTING 資料表 .................................................................................................................... 1068SALES 資料表 ...................................................................................................................... 1068

時區名稱與縮寫 ............................................................................................................................. 1070時區名稱 .............................................................................................................................. 1070時區縮寫 .............................................................................................................................. 1079

文件歷史記錄 ................................................................................................................................ 1083舊版更新 .............................................................................................................................. 1086

xiii

Amazon Redshift 資料庫開發人員指南您是第一次使用 Amazon Redshift 的新手嗎？

Amazon Redshift 系統概觀主題

• 您是第一次使用 Amazon Redshift 的新手嗎？ (p. 1)• 您是資料庫開發人員嗎？ (p. 1)• 先決條件 (p. 2)• 系統和架構概觀 (p. 3)

這是 Amazon Redshift Database Developer Guide。

Amazon Redshift 是一種 PB 級的完全受管企業等級資料倉儲服務。

此指南著重於使用 Amazon Redshift 來建立和管理資料倉儲。若您是以設計人員、軟體開發人員或管理員的身分在使用資料庫，此指南會提供您設計、建置、查詢和維護資料倉儲所需的資訊。

您是第一次使用 Amazon Redshift 的新手嗎？如果您是第一次使用 Amazon Redshift，建議您從閱讀下列章節開始。

• 服務特色和定價：此產品詳細資訊頁面提供 Amazon Redshift 的價值主張、服務特色和定價。• 入門：Amazon Redshift Getting Started 包含範例，可帶您逐步了解建立 Amazon Redshift 資料倉儲叢

集、建立資料庫資料表、上傳資料以及測試查詢的程序。

在您完成入門指南後，建議您探索下列其中一項指南：

• Amazon Redshift Cluster Management Guide：叢集管理指南會示範如何建立及管理 Amazon Redshift 叢集。

若您是應用程式開發人員，可以使用 Amazon Redshift 查詢 API，透過編寫程式的方式來管理叢集。此外，AWS 開發套件程式庫會包裝底層的 Amazon Redshift API，簡化了程式設計任務。若您偏好以更加互動式的方法來管理叢集，可以使用 Amazon Redshift 主控台和 AWS 命令列界面 (AWS CLI)。如需關於API 和 CLI 的資訊，請參閱下列手冊：• API 參考• CLI 參考

• Amazon Redshift Database Developer Guide (此文件)：若您是資料庫開發人員，「資料庫開發人員指南」會說明如何設計、建置、查詢和維護組成資料倉儲的資料庫。

若您是從其他關聯式資料庫系統或資料倉儲應用程式轉移到 Amazon Redshift，請注意 Amazon Redshift 在實作上的重要差異。如需設計資料表和載入資料上最重要考量事項的摘要，請參閱設計資料表的 AmazonRedshift 最佳實務 (p. 19)和載入資料的 Amazon Redshift 最佳實務 (p. 21)。Amazon Redshift 基於PostgreSQL 8.0.2。如需 Amazon Redshift 和 PostgreSQL 之間差異的詳細清單，請參閱 Amazon Redshift與 PostgreSQL (p. 332)。

您是資料庫開發人員嗎？如您是資料庫使用者、資料庫設計人員、資料庫開發人員或資料庫管理員，下表有助於找出您要查詢的項目。

1

https://aws.amazon.com/redshift/https://docs.aws.amazon.com/redshift/latest/gsg/https://docs.aws.amazon.com/redshift/latest/mgmt/https://docs.aws.amazon.com/redshift/latest/APIReference/https://docs.aws.amazon.com/cli/latest/reference/redshift/

Amazon Redshift 資料庫開發人員指南先決條件

如果您想要… 我們建議

使用 Amazon Redshift的快速入門

從遵循 Amazon Redshift Getting Started 中的下列步驟開始，快速部署叢集、連線到資料庫，並嘗試一些查詢。

當您準備好建置資料庫、將資料載入資料表中，以及撰寫查詢以運用資料倉儲中的資料時，請返回此處的「資料庫開發人員指南」。

了解 Amazon Redshift資料倉儲的內部架構。

系統和架構概觀 (p. 3) 顯示 Amazon Redshift 內部架構的高階概觀。

若您想要 Amazon Redshift Web 服務更廣泛的概觀，請前往 Amazon Redshift產品詳細資訊頁面。

建立資料庫、資料表、使用者和其他資料庫物件。

開始使用資料庫 (p. 11) 是 SQL 開發基本知識的快速簡介。

Amazon Redshift SQL (p. 331) 中有 Amazon Redshift SQL 命令和函數，以及其他 SQL 元素的語法和範例。

設計資料表的 Amazon Redshift 最佳實務 (p. 19) 提供了選擇排序索引鍵、散發索引鍵和壓縮編碼上建議的摘要。

了解如何設計資料表以求最佳效能。

設計資料表 (p. 39) 詳述了對資料表資料欄中的資料套用壓縮，以及選擇散發索引鍵和排序索引鍵的考量事項。

載入資料。載入資料 (p. 89) 會說明從 Amazon DynamoDB 資料表或儲存在 AmazonS3 儲存貯體中的一般檔案，載入大型資料集的程序。

載入資料的 Amazon Redshift 最佳實務 (p. 21) 會提供快速有效地載入資料的秘訣。

管理使用者、群組和資料庫安全。

管理資料庫安全 (p. 325) 涵蓋了資料庫安全主題。

監控系統效能並最佳化。

系統資料表參考 (p. 877) 詳述了系統資料表和檢視，可讓您查詢資料庫的狀態，並監控查詢和程序。

您也應參閱 Amazon Redshift Cluster Management Guide 以了解如何使用AWS 管理主控台，來檢查系統運作狀態、監控指標，以及備份和還原叢集。

分析來自非常大型資料集的資訊，並進行報告。

許多受歡迎的軟體廠商會認證 Amazon Redshift 可搭配自家產品和服務，以便讓您繼續使用現在用的工具。如需詳細資訊，請參閱 Amazon Redshift 合作夥伴頁面。

SQL 參考 (p. 331) 中有 Amazon Redshift 所支援的 SQL 表達式、命令和函數相關的所有詳細資訊。

先決條件在使用本指南之前，請先完成下列任務。

• 安裝 SQL 用戶端。• 啟動 Amazon Redshift 叢集。• 將您的 SQL 用戶端連線到叢集主要資料庫。

如需逐步說明，請參閱 Amazon Redshift Getting Started。

您也應了解如何使用 SQL 用戶端，並對 SQL 語言有基本了解。

2

https://docs.aws.amazon.com/redshift/latest/gsg/https://aws.amazon.com/redshift/https://docs.aws.amazon.com/redshift/latest/mgmt/https://aws.amazon.com/redshift/partners/https://aws.amazon.com/redshift/partners/https://docs.aws.amazon.com/redshift/latest/gsg/

Amazon Redshift 資料庫開發人員指南系統和架構概觀

系統和架構概觀主題

• 資料倉儲系統架構 (p. 3)• 效能 (p. 5)• 直欄式儲存 (p. 7)• 內部架構與系統操作 (p. 8)• 工作負載管理 (p. 8)• 搭配其他服務使用 Amazon Redshift (p. 9)

Amazon Redshift 資料倉儲是企業級的關聯式資料庫查詢與管理系統。

Amazon Redshift 透過多種類型的應用程式來支援用戶端連線，包括商業智慧 (BI)、報告、資料與分析工具。

執行分析查詢時，會以多階段操作的方式，擷取、比較和評估大量的資料，來產生最終的結果。

藉由結合大規模的平行處理作業、單欄式資料儲存體，和超高效率的針對式資料壓縮編碼機制，AmazonRedshift 可實現高效率的儲存與最佳化的查詢效能。本節提供 Amazon Redshift 系統架構的介紹。

資料倉儲系統架構本節介紹 Amazon Redshift 資料倉儲架構的元素，如下圖所示。

用戶端應用程式

Amazon Redshift 與各種資料載入和 ETL (擷取、轉換和載入) 工具及商業智慧 (BI) 報告、資料採礦及分析工具整合。Amazon Redshift 採用業界標準 PostgreSQL，因此大多數現有 SQL 用戶端應用程式只要稍微變更即可運作。關於 Amazon Redshift SQL 與 PostgreSQL 之間重要差異的相關資訊，請參閱 Amazon Redshift與 PostgreSQL (p. 332)。

連線

3

Amazon Redshift 資料庫開發人員指南資料倉儲系統架構

Amazon Redshift 使用適用於 PostgreSQL 的業界標準 JDBC 和 ODBC 驅動程式，與用戶端應用程式進行通訊。如需更多詳細資訊，請參閱 Amazon Redshift 和 PostgreSQL JDBC 與 ODBC (p. 332)。

叢集

Amazon Redshift 資料倉儲的核心基礎設施元件是叢集。

叢集是由一個或多個運算節點所組成。如果為叢集佈建了兩個或多個運算節點，會有另外的領導節點負責統籌運算節點和處理外部通訊。您的用戶端應用程式只會直接和領導節點互動，外部應用程式不會知道運算節點的存在。

領導節點

領導節點會管理與用戶端程式的通訊，以及和運算節點的所有通訊。此節點會剖析和制定執行計畫，來執行資料庫的操作，尤其是取得複雜查詢結果所需的一系列步驟。領導節點會根據執行計畫，來編譯程式碼、將編譯過的程式碼分發到運算節點，並指派資料的各部分給每個運算節點。

只有當查詢參考儲存於運算節點上的資料表時，領導節點才會將 SQL 陳述式分發到運算節點。其他所有查詢只在領導節點上執行。Amazon Redshift 設計為只在領導節點上執行某些 SQL 函數。如果查詢使用這些函式的任一個，而且參考位於運算節點上的資料表，將會傳回錯誤。如需更多詳細資訊，請參閱領導節點上所支援的 SQL 函數 (p. 331)。

運算節點

領導節點會針對執行計畫的個別元素，來編譯程式碼，並將程式碼指派給個別的運算節點。運算節點會執行編譯過的程式碼，並將中間的結果傳回領導節點，以進行最後的綜合。

每個運算節點都具有自己專屬的 CPU、記憶體和附加磁碟儲存空間 (根據節點類型決定)。隨著工作負載的增長，您可以藉由增加節點的數量、將節點的類型升級，或是同時實行這兩種做法�

Amazon Redshift - 資料庫開發人員指南...Amazon Redshift 資料庫開發人員指南...

Documents

Transcript of Amazon Redshift - 資料庫開發人員指南...Amazon Redshift 資料庫開發人員指南...