RedisConf17 - Building Large High Performance Redis Databases with Redis Enterprise
Redis tutoring
-
Upload
chen-tien-tsai -
Category
Engineering
-
view
590 -
download
3
Transcript of Redis tutoring
Redis Tutoring
Blackie2015/10/12
Redis IntroductionStackExchange.Redis IntroductionRedisDemo IntroductionTools
Who am I
Blackie TsaiSenior IT consultant of Xuenn
Full stack developerMajor on development of real-time transaction system with low latency and high concurrentLearning CI&CD and run with Agile&LEAN
Bloghttp://www.dotblogs.com.tw/blackie1019
REDIS INTRODUCTION
Redis
Redis - A.K.A Remote Directory Server. It is an open source (BSD licensed), one of Key-Value database of NoSQL, in-memory data structure store, used as database, cache and message broker. It also can run atomic operations.Most Popular NoSQL(http://techstacks.io/)Recommend using Linux for deploying.Features
Pure SimpleSingle TreadIn-memory but persistent on disk databaseRemote dictionary server
3.0.4 is the latest stable version.
開源資料庫Redis實戰經驗大公開
https://www.g2crowd.com/press-release/best-nosql-databases-fall-2015/
https://clusterhq.com/assets/pdfs/state-of-container-usage-june-2015.pdf
Over 70% would like to run a database or other stateful service in their container environments, with MySQL and Redis the two leading choicesImportant features for data management in container solutions were:
Integration of data management capabilities into existing container work flow and toolsseamless movement of data between dev, test and production environments.
Who using Redis
Facebook’s Instagram: Making the Switch to Cassandra from Redis, a 75% ‘Insta’ Savings
http://techstacks.io/tech/redis、http://redis.io/topics/whos-using-redis
http://redis.io/clients
Redis - Data Persistence
MasterRedis
SlaverRedis
Persistence to Disk
Server A
Server B
Replication
IMPLEMENTING PERSISTENCE IN REDIS
• Master instance with no persistence• Slave instance with AOF enabled
Redis - PubSub
SignalR Scaleout with Redis
Redis - Insight Of Pit
Server-side session with RedisRedis has many eviction policies, but most of them are based on 'sampling‘.Alternative Solution
Use Database as an another back-endUse Redis 3.0
Maximize CPUs usageRedis is single thread. One instance usually only use one CPU
Redis, another step on the road
Redis - DataStore
http://redis.io/commands#stringhttp://redis.io/commands#hashhttp://redis.io/commands#sethttp://redis.io/commands#sorted_sethttp://redis.io/commands#list
Redis – LUA Script
Redis embedded scripting languageGood Learning : Lua: 给 Redis 用户的入门指导5 Methods For Tracing and Debugging Redis Lua Scripts
http://redis.io/commands/eval
STACKEXCHANGE.REDIS INTRODUCTION
Stack Exchange
The world’s largest programmingcommunity is growingStack Exchange is a network of 130+ Q&A communities including Stack OverflowGlobal traffic ranking 54th largest website
Architecture of Stack Exchange
http://stackexchange.com/performance
Stack Exchange - Info
Stack Overflow still uses Microsoft products.Stack Overflow still uses a scale-up strategy with HA.SQL Servers loaded with 384 GB of RAM and 2TB of SSD.Stats
4 million users, 8 million questions, 40 million answers, 560 million pageviews a month.Peak is more like 2600-3000 requests/sec on most weekdays. 25 servers, Stack Overflow has a 40:60 read-write ratio. 2 TB of SQL data all stored on SSDs, Each web server has 2x 320GB SSDs in a RAID 1.DB servers average 10% CPU utilization, 11 web servers, using IIS.2 load balancers, 1 active, using HAProxy4 active database nodes, using MS SQL2 machines for distributed cache and messaging using Redis2 read-only SQL Servers for used mainly for the Stack Exchange API3 machines doing search with ElasticSearch
Stack Exchange - Caching
CachingCache all the things.
5 levels of caches.
1st: Caching in the browser, CDN, and proxies.
2nd: Using HttpRuntime.Cache. An in-memory,
per server cache.
3rd: Redis.
4th: SQL Server
Cache. 5th: SSD
.
Stack Exchange - Lessons Learned
Why use Redis if you use MS products? gabeech: It's not about OS evangelism. We run things on the platform they run best on. Period. C# runs best on a windows machine, we use IIS. Redis runs best on a *nix machine we use *nix.
Overkill as a strategySSDs RockKnow your read/write workloadKeeping things very efficient means new machines are not needed oftenDon’t be afraid to specializeDo only what needs to be doneReinvention is OKGo down to the bare metal No bureaucracy.Garbage collection driven programmingThe cost of inefficient code can be higher than you think
StackExchange.Redis
Basic Usage - getting started and basic usageConfiguration - options available when connecting to redisPipelines and Multiplexers - what is a multiplexer?Keys, Values and Channels - discusses the data-types used on the APITransactions - how atomic transactions work in redisEvents - the events available for logging / information purposesPub/Sub Message Order - advice on sequential and concurrent processingScripting - running Lua scripts with convenient named parameter replacement
https://github.com/StackExchange/StackExchange.Redis
How to use
Configuration
Automatic and Manual ConfigurationString Parse Constructor
New Constructor
Configuration OptionsRenaming CommandsTwemproxy
StackExchange.Redis Configuration
REDISDEMO INTRODUCTION
RedisDemo
FeaturesConnection Mapping with ConfigurationConfiguration with Redis Instance Group and Name concept supportedSingleton pattern avoid resource waste
DependencyStackExchange.RedisFX.ConfigurationNewtonsoft.JsonLog4Net(Optional)
Demo Versionhttps://github.com/blackie1019/RedisDemo
RedisDemo - Configuration
ConnectionSetting follow StackExchange.RedisConfigurationOptions.Parse()
RedisDemo Framework - Example
Example Lab
Example – Basic StringsListsSetsHashesSorted Sets
Example – AdvanceSortExpireHashSet, HashGetAll and HashDeletePub/SubPipelinesBatch
Example – Scripting
https://github.com/blackie1019/RedisDemo
TOOLS
Redis Data Management
Cross-platform redis desktop manager - desktop management GUI for mac OS X, Windows, Debian and Ubuntu.
http://redisdesktop.com
Opserver
Monitoring Tool of Stack ExchangeServersSQL clusters/instancesRedisElastic searchException logsHaproxy
Redis Server with Docker
Enable Virtualization Technology on Bios and install Docker ToolboxCreate a Docker container for RedisRun the serviceCreate your web application container
If any problem you can remove and setup againdocker-machine rm defaultdocker-machine --native-ssh create -d virtualbox default
Dockerizing a Redis service
Reference
Scaling Stack Overflow (QCon NYC 2015)Redis, another step on the roadTypes of NoSQL databasesStackOverflow Update: 560M Pageviews A Month, 25 Servers, And It's All About PerformanceRedis 设计与实现《Redis 设计与实现》图片集