RedisGearsndashRedisinmemorydataprocessingJUNE 2019 | PIETER CAILLIAU

bull ProducedinBelgium

bull (instanceof)SETomTom

bull Consultantneo4j

bull SolutionArchitectRedisLabs

bull ProductManagerRedisLabs

bull cailliaup

Aboutme

2

1 WhatisRedisandRedisEnterprise

2 StreamProcessingwithRedisGears

3 RedisGearsasaMultimodelEngine

Agenda

4

RedisisFasthellip

5

hellipExtremelyFast

DB-EnginesRanking

6

7

Andyoursquovebeenusingitalready

RedisisExtensivelyandDiverselyUsed

8

UsesRedisforTimelinefollowing

Scope10-20TB

UsesRedisforLocalsiteglobalcaching

UsesRedisforRepositoryrouter

Scope10+TB

UsesRedisforGeosearchuserprofiles

Scope10-20TB

UsesRedisforAllmessages

Scope40TB

RedisTopDifferentiators

Simplicity ExtensibilityPerformanceNoSQLBenchmark

1

RedisDataStructures

2 3

RedisModules

9

Lists

Hashes

Bitmaps

Strings

Bitfield

Streams

Hyperloglog

SortedSets

Sets

GeospatialIndexes

10

WritteninC

Servedentirelyfrommemory

Single-threadedlockfree

MostcommandsareexecutedwithO(1)

complexity

Accesstodiscreteelements

withinobjects

Reducedbandwidthoverheadrequirements

Easytoparsenetworkingprotocol

Pipeliningforreduced

networkoverhead

Connectionpooling

OPTIMIZEDARCHITECTURE ADVANCEDPROCESSING EFFICIENTOPERATION

RedisSpeeddifferentiators

11

WritteninC

Servedentirelyfrommemory

Single-threadedlockfree

MostcommandsareexecutedwithO(1)

complexity

Accesstodiscreteelements

withinobjects

Reducedbandwidthoverheadrequirements

Easytoparsenetworkingprotocol

Pipeliningforreduced

networkoverhead

Connectionpooling

OPTIMIZEDARCHITECTURE ADVANCEDPROCESSING EFFICIENTOPERATION

RedisSpeeddifferentiators

bull Createyourowndatatypesandcommands

bull ReuseRedisrsquosimplicityperformancescalabilityandhighavailability

bull CanbewritteninCC++GoPythonRustZig

bull Leverageexistingdatastructures

bull TurnRedisintoaMulti-Modeldatabase

ModulesExtendRedisInfinitely

12

httpsredislabscomcommunityredis-modules-hub

RediSearch(GA) redisearchio

RedisBloom(GA) redisbloomio

RedisTimeSeries redistimeseriesio

RedisJSON(GA) redisjsonio

RedisAI redisaiio

RedisGraph(GA) redisgraphio

RedisModules

13

bull ProducedinBelgium

bull (instanceof)SETomTom

bull Consultantneo4j

bull SolutionArchitectRedisLabs

bull ProductManagerRedisLabs

bull cailliaup

Aboutme

2

1 WhatisRedisandRedisEnterprise

2 StreamProcessingwithRedisGears

3 RedisGearsasaMultimodelEngine

Agenda

4

RedisisFasthellip

5

hellipExtremelyFast

DB-EnginesRanking

6

7

Andyoursquovebeenusingitalready

RedisisExtensivelyandDiverselyUsed

8

UsesRedisforTimelinefollowing

Scope10-20TB

UsesRedisforLocalsiteglobalcaching

UsesRedisforRepositoryrouter

Scope10+TB

UsesRedisforGeosearchuserprofiles

Scope10-20TB

UsesRedisforAllmessages

Scope40TB

RedisTopDifferentiators

Simplicity ExtensibilityPerformanceNoSQLBenchmark

1

RedisDataStructures

2 3

RedisModules

9

Lists

Hashes

Bitmaps

Strings

Bitfield

Streams

Hyperloglog

SortedSets

Sets

GeospatialIndexes

10

WritteninC

Servedentirelyfrommemory

Single-threadedlockfree

MostcommandsareexecutedwithO(1)

complexity

Accesstodiscreteelements

withinobjects

Reducedbandwidthoverheadrequirements

Easytoparsenetworkingprotocol

Pipeliningforreduced

networkoverhead

Connectionpooling

OPTIMIZEDARCHITECTURE ADVANCEDPROCESSING EFFICIENTOPERATION

RedisSpeeddifferentiators

11

WritteninC

Servedentirelyfrommemory

Single-threadedlockfree

MostcommandsareexecutedwithO(1)

complexity

Accesstodiscreteelements

withinobjects

Reducedbandwidthoverheadrequirements

Easytoparsenetworkingprotocol

Pipeliningforreduced

networkoverhead

Connectionpooling

OPTIMIZEDARCHITECTURE ADVANCEDPROCESSING EFFICIENTOPERATION

RedisSpeeddifferentiators

bull Createyourowndatatypesandcommands

bull ReuseRedisrsquosimplicityperformancescalabilityandhighavailability

bull CanbewritteninCC++GoPythonRustZig

bull Leverageexistingdatastructures

bull TurnRedisintoaMulti-Modeldatabase

ModulesExtendRedisInfinitely

12

httpsredislabscomcommunityredis-modules-hub

RediSearch(GA) redisearchio

RedisBloom(GA) redisbloomio

RedisTimeSeries redistimeseriesio

RedisJSON(GA) redisjsonio

RedisAI redisaiio

RedisGraph(GA) redisgraphio

RedisModules

13

1 WhatisRedisandRedisEnterprise

2 StreamProcessingwithRedisGears

3 RedisGearsasaMultimodelEngine

Agenda

4

RedisisFasthellip

5

hellipExtremelyFast

DB-EnginesRanking

6

7

Andyoursquovebeenusingitalready

RedisisExtensivelyandDiverselyUsed

8

UsesRedisforTimelinefollowing

Scope10-20TB

UsesRedisforLocalsiteglobalcaching

UsesRedisforRepositoryrouter

Scope10+TB

UsesRedisforGeosearchuserprofiles

Scope10-20TB

UsesRedisforAllmessages

Scope40TB

RedisTopDifferentiators

Simplicity ExtensibilityPerformanceNoSQLBenchmark

1

RedisDataStructures

2 3

RedisModules

9

Lists

Hashes

Bitmaps

Strings

Bitfield

Streams

Hyperloglog

SortedSets

Sets

GeospatialIndexes

10

WritteninC

Servedentirelyfrommemory

Single-threadedlockfree

MostcommandsareexecutedwithO(1)

complexity

Accesstodiscreteelements

withinobjects

Reducedbandwidthoverheadrequirements

Easytoparsenetworkingprotocol

Pipeliningforreduced

networkoverhead

Connectionpooling

OPTIMIZEDARCHITECTURE ADVANCEDPROCESSING EFFICIENTOPERATION

RedisSpeeddifferentiators

11

WritteninC

Servedentirelyfrommemory

Single-threadedlockfree

MostcommandsareexecutedwithO(1)

complexity

Accesstodiscreteelements

withinobjects

Reducedbandwidthoverheadrequirements

Easytoparsenetworkingprotocol

Pipeliningforreduced

networkoverhead

Connectionpooling

OPTIMIZEDARCHITECTURE ADVANCEDPROCESSING EFFICIENTOPERATION

RedisSpeeddifferentiators

bull Createyourowndatatypesandcommands

bull ReuseRedisrsquosimplicityperformancescalabilityandhighavailability

bull CanbewritteninCC++GoPythonRustZig

bull Leverageexistingdatastructures

bull TurnRedisintoaMulti-Modeldatabase

ModulesExtendRedisInfinitely

12

httpsredislabscomcommunityredis-modules-hub

RediSearch(GA) redisearchio

RedisBloom(GA) redisbloomio

RedisTimeSeries redistimeseriesio

RedisJSON(GA) redisjsonio

RedisAI redisaiio

RedisGraph(GA) redisgraphio

RedisModules

13

4

RedisisFasthellip

5

hellipExtremelyFast

DB-EnginesRanking

6

7

Andyoursquovebeenusingitalready

RedisisExtensivelyandDiverselyUsed

8

UsesRedisforTimelinefollowing

Scope10-20TB

UsesRedisforLocalsiteglobalcaching

UsesRedisforRepositoryrouter

Scope10+TB

UsesRedisforGeosearchuserprofiles

Scope10-20TB

UsesRedisforAllmessages

Scope40TB

RedisTopDifferentiators

Simplicity ExtensibilityPerformanceNoSQLBenchmark

1

RedisDataStructures

2 3

RedisModules

9

Lists

Hashes

Bitmaps

Strings

Bitfield

Streams

Hyperloglog

SortedSets

Sets

GeospatialIndexes

10

WritteninC

Servedentirelyfrommemory

Single-threadedlockfree

MostcommandsareexecutedwithO(1)

complexity

Accesstodiscreteelements

withinobjects

Reducedbandwidthoverheadrequirements

Easytoparsenetworkingprotocol

Pipeliningforreduced

networkoverhead

Connectionpooling

OPTIMIZEDARCHITECTURE ADVANCEDPROCESSING EFFICIENTOPERATION

RedisSpeeddifferentiators

11

WritteninC

Servedentirelyfrommemory

Single-threadedlockfree

MostcommandsareexecutedwithO(1)

complexity

Accesstodiscreteelements

withinobjects

Reducedbandwidthoverheadrequirements

Easytoparsenetworkingprotocol

Pipeliningforreduced

networkoverhead

Connectionpooling

OPTIMIZEDARCHITECTURE ADVANCEDPROCESSING EFFICIENTOPERATION

RedisSpeeddifferentiators

bull Createyourowndatatypesandcommands

bull ReuseRedisrsquosimplicityperformancescalabilityandhighavailability

bull CanbewritteninCC++GoPythonRustZig

bull Leverageexistingdatastructures

bull TurnRedisintoaMulti-Modeldatabase

ModulesExtendRedisInfinitely

12

httpsredislabscomcommunityredis-modules-hub

RediSearch(GA) redisearchio

RedisBloom(GA) redisbloomio

RedisTimeSeries redistimeseriesio

RedisJSON(GA) redisjsonio

RedisAI redisaiio

RedisGraph(GA) redisgraphio

RedisModules

13

5

hellipExtremelyFast

DB-EnginesRanking

6

7

Andyoursquovebeenusingitalready

RedisisExtensivelyandDiverselyUsed

8

UsesRedisforTimelinefollowing

Scope10-20TB

UsesRedisforLocalsiteglobalcaching

UsesRedisforRepositoryrouter

Scope10+TB

UsesRedisforGeosearchuserprofiles

Scope10-20TB

UsesRedisforAllmessages

Scope40TB

RedisTopDifferentiators

Simplicity ExtensibilityPerformanceNoSQLBenchmark

1

RedisDataStructures

2 3

RedisModules

9

Lists

Hashes

Bitmaps

Strings

Bitfield

Streams

Hyperloglog

SortedSets

Sets

GeospatialIndexes

10

WritteninC

Servedentirelyfrommemory

Single-threadedlockfree

MostcommandsareexecutedwithO(1)

complexity

Accesstodiscreteelements

withinobjects

Reducedbandwidthoverheadrequirements

Easytoparsenetworkingprotocol

Pipeliningforreduced

networkoverhead

Connectionpooling

OPTIMIZEDARCHITECTURE ADVANCEDPROCESSING EFFICIENTOPERATION

RedisSpeeddifferentiators

11

WritteninC

Servedentirelyfrommemory

Single-threadedlockfree

MostcommandsareexecutedwithO(1)

complexity

Accesstodiscreteelements

withinobjects

Reducedbandwidthoverheadrequirements

Easytoparsenetworkingprotocol

Pipeliningforreduced

networkoverhead

Connectionpooling

OPTIMIZEDARCHITECTURE ADVANCEDPROCESSING EFFICIENTOPERATION

RedisSpeeddifferentiators

bull Createyourowndatatypesandcommands

bull ReuseRedisrsquosimplicityperformancescalabilityandhighavailability

bull CanbewritteninCC++GoPythonRustZig

bull Leverageexistingdatastructures

bull TurnRedisintoaMulti-Modeldatabase

ModulesExtendRedisInfinitely

12

httpsredislabscomcommunityredis-modules-hub

RediSearch(GA) redisearchio

RedisBloom(GA) redisbloomio

RedisTimeSeries redistimeseriesio

RedisJSON(GA) redisjsonio

RedisAI redisaiio

RedisGraph(GA) redisgraphio

RedisModules

13

DB-EnginesRanking

6

7

Andyoursquovebeenusingitalready

RedisisExtensivelyandDiverselyUsed

8

UsesRedisforTimelinefollowing

Scope10-20TB

UsesRedisforLocalsiteglobalcaching

UsesRedisforRepositoryrouter

Scope10+TB

UsesRedisforGeosearchuserprofiles

Scope10-20TB

UsesRedisforAllmessages

Scope40TB

RedisTopDifferentiators

Simplicity ExtensibilityPerformanceNoSQLBenchmark

1

RedisDataStructures

2 3

RedisModules

9

Lists

Hashes

Bitmaps

Strings

Bitfield

Streams

Hyperloglog

SortedSets

Sets

GeospatialIndexes

10

WritteninC

Servedentirelyfrommemory

Single-threadedlockfree

MostcommandsareexecutedwithO(1)

complexity

Accesstodiscreteelements

withinobjects

Reducedbandwidthoverheadrequirements

Easytoparsenetworkingprotocol

Pipeliningforreduced

networkoverhead

Connectionpooling

OPTIMIZEDARCHITECTURE ADVANCEDPROCESSING EFFICIENTOPERATION

RedisSpeeddifferentiators

11

WritteninC

Servedentirelyfrommemory

Single-threadedlockfree

MostcommandsareexecutedwithO(1)

complexity

Accesstodiscreteelements

withinobjects

Reducedbandwidthoverheadrequirements

Easytoparsenetworkingprotocol

Pipeliningforreduced

networkoverhead

Connectionpooling

OPTIMIZEDARCHITECTURE ADVANCEDPROCESSING EFFICIENTOPERATION

RedisSpeeddifferentiators

bull Createyourowndatatypesandcommands

bull ReuseRedisrsquosimplicityperformancescalabilityandhighavailability

bull CanbewritteninCC++GoPythonRustZig

bull Leverageexistingdatastructures

bull TurnRedisintoaMulti-Modeldatabase

ModulesExtendRedisInfinitely

12

httpsredislabscomcommunityredis-modules-hub

RediSearch(GA) redisearchio

RedisBloom(GA) redisbloomio

RedisTimeSeries redistimeseriesio

RedisJSON(GA) redisjsonio

RedisAI redisaiio

RedisGraph(GA) redisgraphio

RedisModules

13

7

Andyoursquovebeenusingitalready

RedisisExtensivelyandDiverselyUsed

8

UsesRedisforTimelinefollowing

Scope10-20TB

UsesRedisforLocalsiteglobalcaching

UsesRedisforRepositoryrouter

Scope10+TB

UsesRedisforGeosearchuserprofiles

Scope10-20TB

UsesRedisforAllmessages

Scope40TB

RedisTopDifferentiators

Simplicity ExtensibilityPerformanceNoSQLBenchmark

1

RedisDataStructures

2 3

RedisModules

9

Lists

Hashes

Bitmaps

Strings

Bitfield

Streams

Hyperloglog

SortedSets

Sets

GeospatialIndexes

10

WritteninC

Servedentirelyfrommemory

Single-threadedlockfree

MostcommandsareexecutedwithO(1)

complexity

Accesstodiscreteelements

withinobjects

Reducedbandwidthoverheadrequirements

Easytoparsenetworkingprotocol

Pipeliningforreduced

networkoverhead

Connectionpooling

OPTIMIZEDARCHITECTURE ADVANCEDPROCESSING EFFICIENTOPERATION

RedisSpeeddifferentiators

11

WritteninC

Servedentirelyfrommemory

Single-threadedlockfree

MostcommandsareexecutedwithO(1)

complexity

Accesstodiscreteelements

withinobjects

Reducedbandwidthoverheadrequirements

Easytoparsenetworkingprotocol

Pipeliningforreduced

networkoverhead

Connectionpooling

OPTIMIZEDARCHITECTURE ADVANCEDPROCESSING EFFICIENTOPERATION

RedisSpeeddifferentiators

bull Createyourowndatatypesandcommands

bull ReuseRedisrsquosimplicityperformancescalabilityandhighavailability

bull CanbewritteninCC++GoPythonRustZig

bull Leverageexistingdatastructures

bull TurnRedisintoaMulti-Modeldatabase

ModulesExtendRedisInfinitely

12

httpsredislabscomcommunityredis-modules-hub

RediSearch(GA) redisearchio

RedisBloom(GA) redisbloomio

RedisTimeSeries redistimeseriesio

RedisJSON(GA) redisjsonio

RedisAI redisaiio

RedisGraph(GA) redisgraphio

RedisModules

13

RedisisExtensivelyandDiverselyUsed

8

UsesRedisforTimelinefollowing

Scope10-20TB

UsesRedisforLocalsiteglobalcaching

UsesRedisforRepositoryrouter

Scope10+TB

UsesRedisforGeosearchuserprofiles

Scope10-20TB

UsesRedisforAllmessages

Scope40TB

RedisTopDifferentiators

Simplicity ExtensibilityPerformanceNoSQLBenchmark

1

RedisDataStructures

2 3

RedisModules

9

Lists

Hashes

Bitmaps

Strings

Bitfield

Streams

Hyperloglog

SortedSets

Sets

GeospatialIndexes

10

WritteninC

Servedentirelyfrommemory

Single-threadedlockfree

MostcommandsareexecutedwithO(1)

complexity

Accesstodiscreteelements

withinobjects

Reducedbandwidthoverheadrequirements

Easytoparsenetworkingprotocol

Pipeliningforreduced

networkoverhead

Connectionpooling

OPTIMIZEDARCHITECTURE ADVANCEDPROCESSING EFFICIENTOPERATION

RedisSpeeddifferentiators

11

WritteninC

Servedentirelyfrommemory

Single-threadedlockfree

MostcommandsareexecutedwithO(1)

complexity

Accesstodiscreteelements

withinobjects

Reducedbandwidthoverheadrequirements

Easytoparsenetworkingprotocol

Pipeliningforreduced

networkoverhead

Connectionpooling

OPTIMIZEDARCHITECTURE ADVANCEDPROCESSING EFFICIENTOPERATION

RedisSpeeddifferentiators

bull Createyourowndatatypesandcommands

bull ReuseRedisrsquosimplicityperformancescalabilityandhighavailability

bull CanbewritteninCC++GoPythonRustZig

bull Leverageexistingdatastructures

bull TurnRedisintoaMulti-Modeldatabase

ModulesExtendRedisInfinitely

12

httpsredislabscomcommunityredis-modules-hub

RediSearch(GA) redisearchio

RedisBloom(GA) redisbloomio

RedisTimeSeries redistimeseriesio

RedisJSON(GA) redisjsonio

RedisAI redisaiio

RedisGraph(GA) redisgraphio

RedisModules

13

RedisTopDifferentiators

Simplicity ExtensibilityPerformanceNoSQLBenchmark

1

RedisDataStructures

2 3

RedisModules

9

Lists

Hashes

Bitmaps

Strings

Bitfield

Streams

Hyperloglog

SortedSets

Sets

GeospatialIndexes

10

WritteninC

Servedentirelyfrommemory

Single-threadedlockfree

MostcommandsareexecutedwithO(1)

complexity

Accesstodiscreteelements

withinobjects

Reducedbandwidthoverheadrequirements

Easytoparsenetworkingprotocol

Pipeliningforreduced

networkoverhead

Connectionpooling

OPTIMIZEDARCHITECTURE ADVANCEDPROCESSING EFFICIENTOPERATION

RedisSpeeddifferentiators

11

WritteninC

Servedentirelyfrommemory

Single-threadedlockfree

MostcommandsareexecutedwithO(1)

complexity

Accesstodiscreteelements

withinobjects

Reducedbandwidthoverheadrequirements

Easytoparsenetworkingprotocol

Pipeliningforreduced

networkoverhead

Connectionpooling

OPTIMIZEDARCHITECTURE ADVANCEDPROCESSING EFFICIENTOPERATION

RedisSpeeddifferentiators

bull Createyourowndatatypesandcommands

bull ReuseRedisrsquosimplicityperformancescalabilityandhighavailability

bull CanbewritteninCC++GoPythonRustZig

bull Leverageexistingdatastructures

bull TurnRedisintoaMulti-Modeldatabase

ModulesExtendRedisInfinitely

12

httpsredislabscomcommunityredis-modules-hub

RediSearch(GA) redisearchio

RedisBloom(GA) redisbloomio

RedisTimeSeries redistimeseriesio

RedisJSON(GA) redisjsonio

RedisAI redisaiio

RedisGraph(GA) redisgraphio

RedisModules

13

10

WritteninC

Servedentirelyfrommemory

Single-threadedlockfree

MostcommandsareexecutedwithO(1)

complexity

Accesstodiscreteelements

withinobjects

Reducedbandwidthoverheadrequirements

Easytoparsenetworkingprotocol

Pipeliningforreduced

networkoverhead

Connectionpooling

OPTIMIZEDARCHITECTURE ADVANCEDPROCESSING EFFICIENTOPERATION

RedisSpeeddifferentiators

11

WritteninC

Servedentirelyfrommemory

Single-threadedlockfree

MostcommandsareexecutedwithO(1)

complexity

Accesstodiscreteelements

withinobjects

Reducedbandwidthoverheadrequirements

Easytoparsenetworkingprotocol

Pipeliningforreduced

networkoverhead

Connectionpooling

OPTIMIZEDARCHITECTURE ADVANCEDPROCESSING EFFICIENTOPERATION

RedisSpeeddifferentiators

bull Createyourowndatatypesandcommands

bull ReuseRedisrsquosimplicityperformancescalabilityandhighavailability

bull CanbewritteninCC++GoPythonRustZig

bull Leverageexistingdatastructures

bull TurnRedisintoaMulti-Modeldatabase

ModulesExtendRedisInfinitely

12

httpsredislabscomcommunityredis-modules-hub

RediSearch(GA) redisearchio

RedisBloom(GA) redisbloomio

RedisTimeSeries redistimeseriesio

RedisJSON(GA) redisjsonio

RedisAI redisaiio

RedisGraph(GA) redisgraphio

RedisModules

13

11

WritteninC

Servedentirelyfrommemory

Single-threadedlockfree

MostcommandsareexecutedwithO(1)

complexity

Accesstodiscreteelements

withinobjects

Reducedbandwidthoverheadrequirements

Easytoparsenetworkingprotocol

Pipeliningforreduced

networkoverhead

Connectionpooling

OPTIMIZEDARCHITECTURE ADVANCEDPROCESSING EFFICIENTOPERATION

RedisSpeeddifferentiators

bull Createyourowndatatypesandcommands

bull ReuseRedisrsquosimplicityperformancescalabilityandhighavailability

bull CanbewritteninCC++GoPythonRustZig

bull Leverageexistingdatastructures

bull TurnRedisintoaMulti-Modeldatabase

ModulesExtendRedisInfinitely

12

httpsredislabscomcommunityredis-modules-hub

RediSearch(GA) redisearchio

RedisBloom(GA) redisbloomio

RedisTimeSeries redistimeseriesio

RedisJSON(GA) redisjsonio

RedisAI redisaiio

RedisGraph(GA) redisgraphio

RedisModules

13

bull Createyourowndatatypesandcommands

bull ReuseRedisrsquosimplicityperformancescalabilityandhighavailability

bull CanbewritteninCC++GoPythonRustZig

bull Leverageexistingdatastructures

bull TurnRedisintoaMulti-Modeldatabase

ModulesExtendRedisInfinitely

12

httpsredislabscomcommunityredis-modules-hub

RediSearch(GA) redisearchio

RedisBloom(GA) redisbloomio

RedisTimeSeries redistimeseriesio

RedisJSON(GA) redisjsonio

RedisAI redisaiio

RedisGraph(GA) redisgraphio

RedisModules

13

RediSearch(GA) redisearchio

RedisBloom(GA) redisbloomio

RedisTimeSeries redistimeseriesio

RedisJSON(GA) redisjsonio

RedisAI redisaiio

RedisGraph(GA) redisgraphio

RedisModules

13

14

Introducing

RedisEnterprise

15

DBaaSbull Availablesincemid2013

bull 8500+enterprisecustomers

Softwarebull Availablesinceearly2015

bull 300+enterprisecustomers550K+databases

managedworldwide

bull 6oftopFortune10companies

bull 3oftop5communicationscompanies

Customersbull 3oftop4creditcardissuers

bull 3oftop5healthcarecompanies

bull CloudProvider

bull Highermarginbyndash Idleness

bull CloudLock-in

CloudProvidershavedifferentincentives

16

bull DBAASProvider

bull Highermarginbybetterresourceutilizationndash Multi-tenancyndash ReducingRAMndash CPUutilization

RedisEnterpriseAUniquePrimaryDatabase

HIGHESTPERFORMANCELINEARSCALING

HIGHAVAILABILITYWITHINSTANTFAILOVER

DURABILITYATMEMORYSPEEDS

ACTIVE-ACTIVEGEODISTRIBUTION(CRDT-BASED)

BUILT-INHIGHPERFORMANCE SEARCH

MULTI-MODEL

FLEXIBLEDEPLOYMENTOPTIONS(CLOUDON-PREMHYBRID)

INTELLIGENTTIEREDDATAACCESS(RAMampFLASHMEMORY)

FAST RELIABLE FLEXIBLE

RedisEnterpriseCluster

Node1 Node2 NodeN(oddnumber)

18

Unevennumberofsymmetricnodes

RedisEnterpriseCluster

Node1 Node2 NodeN(oddnumber)

19

Singlemasterdatabase

M

RedisEnterpriseCluster

Node1 Node2 NodeN(oddnumber)

20

AnHAdatabase

SM

RedisEnterpriseCluster

Node1 Node2 NodeN(oddnumber)

21

AClusteredDatabase

M1 M2 M3

Howdokeysgetassignedtopartitions

22

Howdokeysgetassignedtopartitions

23

RedisEnterpriseCluster

Node1 Node2 NodeN(oddnumber)

24

AHighlyAvailableClusteredDatabase

M1 M2 M3S3 S1 S2

RedisEnterpriseNode

25

ClusterManager

EnterpriseLayer

OpenSourceLayer

RESTAPIZerolatencyproxy

RedisShards

RedisEnterpriseSharedNothingSymmetricArchitecture

ClusterManagementPath

NodeWatchdogClusterWatchdog

Node1 Node2 NodeN(oddnumber)

RedisShardsampProxies

DataPath

26

Data-PathandControlManagementPathSeparation

27

asadatagrid

MicroservicesArchitectureandPolyglotPersistence

28

AuthenticationAuthentication

Authentication

KeyValue

K V

KeyValue

K V

KeyValue

K V

Customers

KeyValue

K V

Graph

CustomersKeyValue

K V

Graph

Catalog

RDBMS Cache

Search

SessionStore

SessionStore

SessionStore

Document

Document

Document

API

API

API

API

API

API

API

API

API

FraudDetection

API

ColumnarSearch

Fulfilment API

RDBMS

TheCostofPolyglotPersistence

AuthenticationAuthentication

Authentication

KeyValue

K V

KeyValue

K V

KeyValue

K V

Customers

KeyValue

K V

Graph

CustomersKeyValue

K V

Graph

Catalog

RDBMS Cache

Search

SessionStore

SessionStore

SessionStore

Document

Document

Document

API

API

API

API

API

API

API

API

API

FraudDetection

API

ColumnarSearch

Fulfilment API

RDBMS

Increasedapplicationcomplexity CostlycommunicationApplicationdoesheavyliftinginsharingdatakeepingdatasetsinsync

HighoperationalburdenHighercostofownershipDifferentdatabaseshavespecializedadministrativescalingavailability

requirements

Sub-optimalResourceUsage HighercostDedicatingpodsserversforeachtypeofdatabasereducesdeploymentefficiency

API API API API APIAPI API

RedisEnterpriseAMulti-modelDatabaseforMicroservices

Authentication Customers Catalog Search FraudDetection SessionStore

SearchGraphKeyValue

RDBMS

KeyValueCache

RDBMS

Cache

Fulfilment

KeyValue Document

Authentication Customers Catalog Search FraudDetection SessionStore

SearchGraphKeyValue

RDBMS

KeyValueCache

RDBMS

Cache

Fulfilment

KeyValue Document

Built-inMessageBroker

Built-inPub-SubStreamsforeventsynchacrossdatastores

Whatarewemissing

bull Howtoconsumemessagesinthisldquobuiltinmessageborkerrdquo

bull GivenashardeddatabasehowcanIrunanalyticalqueries

bull MultiModeldatabasendash Singlecopyincoredatatypesndash Intermodulecommunicationndash ComponentXdoingtranslationsbetweenmodules

33

RedisGears

Introducing

RedisGearsisaServerlessengineformulti-modelandclusteroperationsinRedissupportingbotheventdriven

aswellasbatchoperations

WhatisRedisGears

GearsCoordinator MapReducerGearsExecuter

GearsinfrastructureiswritteninC

C-API

Soon Soon

HighPerformanceArchitecture

RedisGearsallowstodefineapipeofoperations

bull Returningvaluefromoneoperationispassedtothenextoperationinthepipe

bull Lastoperationreturningtheresulttotheuserbull Firstoperationiscalledlsquoreaderrsquo-responsibleforprovidingdata

ndash Keysreader-readkeysfromRedisndash Streamreader-readstreamsfromRedisndash Pythonreader-allowtousertowritehisownreadersinpython

ScriptingwithRedisGears

36

bull Map

bull FlatMap

bull Filterbull Groupby+Reducebull Aggregatebull Sortbull Limit

bull ForEachbull Distinct

SupportedOperations

37

Reader(Flat)Mapper

UsingRedisGearsndash(Flat)Mapping

UsingRedisGears-Filtering

Reader Filterrecordwith1doc

UsingRedisGears-Aggregate

Reader CountAggregator

1

1

Demo

GearshasastreamingAPItoallowtotriggergearsexecutiononeventsndash RedisStreamevents-Triggeranexecutionwheneveranewdataentersasteamndash RedisKeysevents-Triggeranoperationwheneverakeyistouched

UseCase1ndashStreamProcessing

RedisTimeSeriesRedisStreams

Everysec

BecauseofRedisGearsrsquoflexibility(itsactuallyrunningpython)youcanachieveinternalmoduleintegrationwithitndash ReadfromhashesandindexinRediSearchRedisGraphndash ReadRedisJSONdataandpasstoRedisTimeSeriesndash hellip

UseCase2ndashaMultiModelEngine

RediSearch

RedisHashesEveryupdate

RedisGraph

createthebuilderbuilder=GearsBuilder()filtereventsonkeyall_keysbuilderfilter(lambdaxx[key]=all_keys)addthekeystoall_keyssetbuildermap(lambdaxexecute(saddall_keysx[key]))registertheexecutiononkeyspacenotificationbuilderregister()

Buildagearthatcreatesmaintainsasetofallkeyswithinredis

Recipe1ndasheventriggering

createthepipebuilderKeysOnlyReaderisaperformanceimprovementonlypipingthekeysbuilder=GearsBuilder(KeysOnlyReader)getfromeachhashthegenresfieldbuildermap(lambdaxexecute(hgetxgenres))filterthosewhodonothavegenresbuilderfilter(lambdaxxisnotNone)splitgenresbycommabuilderflatmap(lambdaxxsplit())countforeachgenrethenumberoftimesitappearsbuildercountby()starttheexecutionbuilderrun(movie)

Buildagearthatcountshowoftenagenreisusedwithinasetofmovies

Recipe2ndashmapreducing

createthebuilderwithaStreamReaderbuilder=GearsBuilder(StreamReader)extracteachfieldvaluepairfromthemessageandincreasethepipegranularitybuilderflatmap(lambdax[(a[0]a[1])forainxitems()])filteroutthestreamIditselfbuilderfilter(lambdaxx[0]=streamId)makesurethegearsdatalivesinthecorrectshardbuilderrepartition(lambdaxx[0])applyeachfieldvaluepairtoakeybuilderforeach(lambdaxexecute(setx[0]x[1]))registeronnewmessagesonthestreaminputStreambuilderregister(inputStream)

Buildagearthatconsumesastreamandupdateskeysaccordingly

Recipe3ndashstreamprocessing

ExampleTriggerExplained

47

ExampleTriggerExplained

48

ExampleTriggerExplained

49

ExampleTriggerExplained-Flatmap

50

ExampleTriggerExplained-Repartition

51

ExampleTriggerExplained-executeCommand

52

Demo

DemoSetup

54

bull httpsgithubcomRedisGearsAnimalRecognitionDemo

bull redisfoundmycat

Challenge

55

RedisEnterprise

15

DBaaSbull Availablesincemid2013

bull 8500+enterprisecustomers

Softwarebull Availablesinceearly2015

bull 300+enterprisecustomers550K+databases

managedworldwide

bull 6oftopFortune10companies

bull 3oftop5communicationscompanies

Customersbull 3oftop4creditcardissuers

bull 3oftop5healthcarecompanies

bull CloudProvider

bull Highermarginbyndash Idleness

bull CloudLock-in

CloudProvidershavedifferentincentives

16

bull DBAASProvider

bull Highermarginbybetterresourceutilizationndash Multi-tenancyndash ReducingRAMndash CPUutilization

RedisEnterpriseAUniquePrimaryDatabase

HIGHESTPERFORMANCELINEARSCALING

HIGHAVAILABILITYWITHINSTANTFAILOVER

DURABILITYATMEMORYSPEEDS

ACTIVE-ACTIVEGEODISTRIBUTION(CRDT-BASED)

BUILT-INHIGHPERFORMANCE SEARCH

MULTI-MODEL

FLEXIBLEDEPLOYMENTOPTIONS(CLOUDON-PREMHYBRID)

INTELLIGENTTIEREDDATAACCESS(RAMampFLASHMEMORY)

FAST RELIABLE FLEXIBLE

RedisEnterpriseCluster

Node1 Node2 NodeN(oddnumber)

18

Unevennumberofsymmetricnodes

RedisEnterpriseCluster

Node1 Node2 NodeN(oddnumber)

19

Singlemasterdatabase

M

RedisEnterpriseCluster

Node1 Node2 NodeN(oddnumber)

20

AnHAdatabase

SM

RedisEnterpriseCluster

Node1 Node2 NodeN(oddnumber)

21

AClusteredDatabase

M1 M2 M3

Howdokeysgetassignedtopartitions

22

Howdokeysgetassignedtopartitions

23

RedisEnterpriseCluster

Node1 Node2 NodeN(oddnumber)

24

AHighlyAvailableClusteredDatabase

M1 M2 M3S3 S1 S2

RedisEnterpriseNode

25

ClusterManager

EnterpriseLayer

OpenSourceLayer

RESTAPIZerolatencyproxy

RedisShards

RedisEnterpriseSharedNothingSymmetricArchitecture

ClusterManagementPath

NodeWatchdogClusterWatchdog

Node1 Node2 NodeN(oddnumber)

RedisShardsampProxies

DataPath

26

Data-PathandControlManagementPathSeparation

27

asadatagrid

MicroservicesArchitectureandPolyglotPersistence

28

AuthenticationAuthentication

Authentication

KeyValue

K V

KeyValue

K V

KeyValue

K V

Customers

KeyValue

K V

Graph

CustomersKeyValue

K V

Graph

Catalog

RDBMS Cache

Search

SessionStore

SessionStore

SessionStore

Document

Document

Document

API

API

API

API

API

API

API

API

API

FraudDetection

API

ColumnarSearch

Fulfilment API

RDBMS

TheCostofPolyglotPersistence

AuthenticationAuthentication

Authentication

KeyValue

K V

KeyValue

K V

KeyValue

K V

Customers

KeyValue

K V

Graph

CustomersKeyValue

K V

Graph

Catalog

RDBMS Cache

Search

SessionStore

SessionStore

SessionStore

Document

Document

Document

API

API

API

API

API

API

API

API

API

FraudDetection

API

ColumnarSearch

Fulfilment API

RDBMS

Increasedapplicationcomplexity CostlycommunicationApplicationdoesheavyliftinginsharingdatakeepingdatasetsinsync

HighoperationalburdenHighercostofownershipDifferentdatabaseshavespecializedadministrativescalingavailability

requirements

Sub-optimalResourceUsage HighercostDedicatingpodsserversforeachtypeofdatabasereducesdeploymentefficiency

API API API API APIAPI API

RedisEnterpriseAMulti-modelDatabaseforMicroservices

Authentication Customers Catalog Search FraudDetection SessionStore

SearchGraphKeyValue

RDBMS

KeyValueCache

RDBMS

Cache

Fulfilment

KeyValue Document

Authentication Customers Catalog Search FraudDetection SessionStore

SearchGraphKeyValue

RDBMS

KeyValueCache

RDBMS

Cache

Fulfilment

KeyValue Document

Built-inMessageBroker

Built-inPub-SubStreamsforeventsynchacrossdatastores

Whatarewemissing

bull Howtoconsumemessagesinthisldquobuiltinmessageborkerrdquo

bull GivenashardeddatabasehowcanIrunanalyticalqueries

bull MultiModeldatabasendash Singlecopyincoredatatypesndash Intermodulecommunicationndash ComponentXdoingtranslationsbetweenmodules

33

RedisGears

Introducing

RedisGearsisaServerlessengineformulti-modelandclusteroperationsinRedissupportingbotheventdriven

aswellasbatchoperations

WhatisRedisGears

GearsCoordinator MapReducerGearsExecuter

GearsinfrastructureiswritteninC

C-API

Soon Soon

HighPerformanceArchitecture

RedisGearsallowstodefineapipeofoperations

bull Returningvaluefromoneoperationispassedtothenextoperationinthepipe

bull Lastoperationreturningtheresulttotheuserbull Firstoperationiscalledlsquoreaderrsquo-responsibleforprovidingdata

ndash Keysreader-readkeysfromRedisndash Streamreader-readstreamsfromRedisndash Pythonreader-allowtousertowritehisownreadersinpython

ScriptingwithRedisGears

36

bull Map

bull FlatMap

bull Filterbull Groupby+Reducebull Aggregatebull Sortbull Limit

bull ForEachbull Distinct

SupportedOperations

37

Reader(Flat)Mapper

UsingRedisGearsndash(Flat)Mapping

UsingRedisGears-Filtering

Reader Filterrecordwith1doc

UsingRedisGears-Aggregate

Reader CountAggregator

1

1

Demo

GearshasastreamingAPItoallowtotriggergearsexecutiononeventsndash RedisStreamevents-Triggeranexecutionwheneveranewdataentersasteamndash RedisKeysevents-Triggeranoperationwheneverakeyistouched

UseCase1ndashStreamProcessing

RedisTimeSeriesRedisStreams

Everysec

BecauseofRedisGearsrsquoflexibility(itsactuallyrunningpython)youcanachieveinternalmoduleintegrationwithitndash ReadfromhashesandindexinRediSearchRedisGraphndash ReadRedisJSONdataandpasstoRedisTimeSeriesndash hellip

UseCase2ndashaMultiModelEngine

RediSearch

RedisHashesEveryupdate

RedisGraph

createthebuilderbuilder=GearsBuilder()filtereventsonkeyall_keysbuilderfilter(lambdaxx[key]=all_keys)addthekeystoall_keyssetbuildermap(lambdaxexecute(saddall_keysx[key]))registertheexecutiononkeyspacenotificationbuilderregister()

Buildagearthatcreatesmaintainsasetofallkeyswithinredis

Recipe1ndasheventriggering

createthepipebuilderKeysOnlyReaderisaperformanceimprovementonlypipingthekeysbuilder=GearsBuilder(KeysOnlyReader)getfromeachhashthegenresfieldbuildermap(lambdaxexecute(hgetxgenres))filterthosewhodonothavegenresbuilderfilter(lambdaxxisnotNone)splitgenresbycommabuilderflatmap(lambdaxxsplit())countforeachgenrethenumberoftimesitappearsbuildercountby()starttheexecutionbuilderrun(movie)

Buildagearthatcountshowoftenagenreisusedwithinasetofmovies

Recipe2ndashmapreducing

createthebuilderwithaStreamReaderbuilder=GearsBuilder(StreamReader)extracteachfieldvaluepairfromthemessageandincreasethepipegranularitybuilderflatmap(lambdax[(a[0]a[1])forainxitems()])filteroutthestreamIditselfbuilderfilter(lambdaxx[0]=streamId)makesurethegearsdatalivesinthecorrectshardbuilderrepartition(lambdaxx[0])applyeachfieldvaluepairtoakeybuilderforeach(lambdaxexecute(setx[0]x[1]))registeronnewmessagesonthestreaminputStreambuilderregister(inputStream)

Buildagearthatconsumesastreamandupdateskeysaccordingly

Recipe3ndashstreamprocessing

ExampleTriggerExplained

47

ExampleTriggerExplained

48

ExampleTriggerExplained

49

ExampleTriggerExplained-Flatmap

50

ExampleTriggerExplained-Repartition

51

ExampleTriggerExplained-executeCommand

52

Demo

DemoSetup

54

bull httpsgithubcomRedisGearsAnimalRecognitionDemo

bull redisfoundmycat

Challenge

55

bull CloudProvider

bull Highermarginbyndash Idleness

bull CloudLock-in

CloudProvidershavedifferentincentives

16

bull DBAASProvider

bull Highermarginbybetterresourceutilizationndash Multi-tenancyndash ReducingRAMndash CPUutilization

RedisEnterpriseAUniquePrimaryDatabase

HIGHESTPERFORMANCELINEARSCALING

HIGHAVAILABILITYWITHINSTANTFAILOVER

DURABILITYATMEMORYSPEEDS

ACTIVE-ACTIVEGEODISTRIBUTION(CRDT-BASED)

BUILT-INHIGHPERFORMANCE SEARCH

MULTI-MODEL

FLEXIBLEDEPLOYMENTOPTIONS(CLOUDON-PREMHYBRID)

INTELLIGENTTIEREDDATAACCESS(RAMampFLASHMEMORY)

FAST RELIABLE FLEXIBLE

RedisEnterpriseCluster

Node1 Node2 NodeN(oddnumber)

18

Unevennumberofsymmetricnodes

RedisEnterpriseCluster

Node1 Node2 NodeN(oddnumber)

19

Singlemasterdatabase

M

RedisEnterpriseCluster

Node1 Node2 NodeN(oddnumber)

20

AnHAdatabase

SM

RedisEnterpriseCluster

Node1 Node2 NodeN(oddnumber)

21

AClusteredDatabase

M1 M2 M3

Howdokeysgetassignedtopartitions

22

Howdokeysgetassignedtopartitions

23

RedisEnterpriseCluster

Node1 Node2 NodeN(oddnumber)

24

AHighlyAvailableClusteredDatabase

M1 M2 M3S3 S1 S2

RedisEnterpriseNode

25

ClusterManager

EnterpriseLayer

OpenSourceLayer

RESTAPIZerolatencyproxy

RedisShards

RedisEnterpriseSharedNothingSymmetricArchitecture

ClusterManagementPath

NodeWatchdogClusterWatchdog

Node1 Node2 NodeN(oddnumber)

RedisShardsampProxies

DataPath

26

Data-PathandControlManagementPathSeparation

27

asadatagrid

MicroservicesArchitectureandPolyglotPersistence

28

AuthenticationAuthentication

Authentication

KeyValue

K V

KeyValue

K V

KeyValue

K V

Customers

KeyValue

K V

Graph

CustomersKeyValue

K V

Graph

Catalog

RDBMS Cache

Search

SessionStore

SessionStore

SessionStore

Document

Document

Document

API

API

API

API

API

API

API

API

API

FraudDetection

API

ColumnarSearch

Fulfilment API

RDBMS

TheCostofPolyglotPersistence

AuthenticationAuthentication

Authentication

KeyValue

K V

KeyValue

K V

KeyValue

K V

Customers

KeyValue

K V

Graph

CustomersKeyValue

K V

Graph

Catalog

RDBMS Cache

Search

SessionStore

SessionStore

SessionStore

Document

Document

Document

API

API

API

API

API

API

API

API

API

FraudDetection

API

ColumnarSearch

Fulfilment API

RDBMS

Increasedapplicationcomplexity CostlycommunicationApplicationdoesheavyliftinginsharingdatakeepingdatasetsinsync

HighoperationalburdenHighercostofownershipDifferentdatabaseshavespecializedadministrativescalingavailability

requirements

Sub-optimalResourceUsage HighercostDedicatingpodsserversforeachtypeofdatabasereducesdeploymentefficiency

API API API API APIAPI API

RedisEnterpriseAMulti-modelDatabaseforMicroservices

Authentication Customers Catalog Search FraudDetection SessionStore

SearchGraphKeyValue

RDBMS

KeyValueCache

RDBMS

Cache

Fulfilment

KeyValue Document

Authentication Customers Catalog Search FraudDetection SessionStore

SearchGraphKeyValue

RDBMS

KeyValueCache

RDBMS

Cache

Fulfilment

KeyValue Document

Built-inMessageBroker

Built-inPub-SubStreamsforeventsynchacrossdatastores

Whatarewemissing

bull Howtoconsumemessagesinthisldquobuiltinmessageborkerrdquo

bull GivenashardeddatabasehowcanIrunanalyticalqueries

bull MultiModeldatabasendash Singlecopyincoredatatypesndash Intermodulecommunicationndash ComponentXdoingtranslationsbetweenmodules

33

RedisGears

Introducing

RedisGearsisaServerlessengineformulti-modelandclusteroperationsinRedissupportingbotheventdriven

aswellasbatchoperations

WhatisRedisGears

GearsCoordinator MapReducerGearsExecuter

GearsinfrastructureiswritteninC

C-API

Soon Soon

HighPerformanceArchitecture

RedisGearsallowstodefineapipeofoperations

bull Returningvaluefromoneoperationispassedtothenextoperationinthepipe

bull Lastoperationreturningtheresulttotheuserbull Firstoperationiscalledlsquoreaderrsquo-responsibleforprovidingdata

ndash Keysreader-readkeysfromRedisndash Streamreader-readstreamsfromRedisndash Pythonreader-allowtousertowritehisownreadersinpython

ScriptingwithRedisGears

36

bull Map

bull FlatMap

bull Filterbull Groupby+Reducebull Aggregatebull Sortbull Limit

bull ForEachbull Distinct

SupportedOperations

37

Reader(Flat)Mapper

UsingRedisGearsndash(Flat)Mapping

UsingRedisGears-Filtering

Reader Filterrecordwith1doc

UsingRedisGears-Aggregate

Reader CountAggregator

1

1

Demo

GearshasastreamingAPItoallowtotriggergearsexecutiononeventsndash RedisStreamevents-Triggeranexecutionwheneveranewdataentersasteamndash RedisKeysevents-Triggeranoperationwheneverakeyistouched

UseCase1ndashStreamProcessing

RedisTimeSeriesRedisStreams

Everysec

BecauseofRedisGearsrsquoflexibility(itsactuallyrunningpython)youcanachieveinternalmoduleintegrationwithitndash ReadfromhashesandindexinRediSearchRedisGraphndash ReadRedisJSONdataandpasstoRedisTimeSeriesndash hellip

UseCase2ndashaMultiModelEngine

RediSearch

RedisHashesEveryupdate

RedisGraph

createthebuilderbuilder=GearsBuilder()filtereventsonkeyall_keysbuilderfilter(lambdaxx[key]=all_keys)addthekeystoall_keyssetbuildermap(lambdaxexecute(saddall_keysx[key]))registertheexecutiononkeyspacenotificationbuilderregister()

Buildagearthatcreatesmaintainsasetofallkeyswithinredis

Recipe1ndasheventriggering

createthepipebuilderKeysOnlyReaderisaperformanceimprovementonlypipingthekeysbuilder=GearsBuilder(KeysOnlyReader)getfromeachhashthegenresfieldbuildermap(lambdaxexecute(hgetxgenres))filterthosewhodonothavegenresbuilderfilter(lambdaxxisnotNone)splitgenresbycommabuilderflatmap(lambdaxxsplit())countforeachgenrethenumberoftimesitappearsbuildercountby()starttheexecutionbuilderrun(movie)

Buildagearthatcountshowoftenagenreisusedwithinasetofmovies

Recipe2ndashmapreducing

createthebuilderwithaStreamReaderbuilder=GearsBuilder(StreamReader)extracteachfieldvaluepairfromthemessageandincreasethepipegranularitybuilderflatmap(lambdax[(a[0]a[1])forainxitems()])filteroutthestreamIditselfbuilderfilter(lambdaxx[0]=streamId)makesurethegearsdatalivesinthecorrectshardbuilderrepartition(lambdaxx[0])applyeachfieldvaluepairtoakeybuilderforeach(lambdaxexecute(setx[0]x[1]))registeronnewmessagesonthestreaminputStreambuilderregister(inputStream)

Buildagearthatconsumesastreamandupdateskeysaccordingly

Recipe3ndashstreamprocessing

ExampleTriggerExplained

47

ExampleTriggerExplained

48

ExampleTriggerExplained

49

ExampleTriggerExplained-Flatmap

50

ExampleTriggerExplained-Repartition

51

ExampleTriggerExplained-executeCommand

52

Demo

DemoSetup

54

bull httpsgithubcomRedisGearsAnimalRecognitionDemo

bull redisfoundmycat

Challenge

55

RedisEnterpriseAUniquePrimaryDatabase

HIGHESTPERFORMANCELINEARSCALING

HIGHAVAILABILITYWITHINSTANTFAILOVER

DURABILITYATMEMORYSPEEDS

ACTIVE-ACTIVEGEODISTRIBUTION(CRDT-BASED)

BUILT-INHIGHPERFORMANCE SEARCH

MULTI-MODEL

FLEXIBLEDEPLOYMENTOPTIONS(CLOUDON-PREMHYBRID)

INTELLIGENTTIEREDDATAACCESS(RAMampFLASHMEMORY)

FAST RELIABLE FLEXIBLE

RedisEnterpriseCluster

Node1 Node2 NodeN(oddnumber)

18

Unevennumberofsymmetricnodes

RedisEnterpriseCluster

Node1 Node2 NodeN(oddnumber)

19

Singlemasterdatabase

M

RedisEnterpriseCluster

Node1 Node2 NodeN(oddnumber)

20

AnHAdatabase

SM

RedisEnterpriseCluster

Node1 Node2 NodeN(oddnumber)

21

AClusteredDatabase

M1 M2 M3

Howdokeysgetassignedtopartitions

22

Howdokeysgetassignedtopartitions

23

RedisEnterpriseCluster

Node1 Node2 NodeN(oddnumber)

24

AHighlyAvailableClusteredDatabase

M1 M2 M3S3 S1 S2

RedisEnterpriseNode

25

ClusterManager

EnterpriseLayer

OpenSourceLayer

RESTAPIZerolatencyproxy

RedisShards

RedisEnterpriseSharedNothingSymmetricArchitecture

ClusterManagementPath

NodeWatchdogClusterWatchdog

Node1 Node2 NodeN(oddnumber)

RedisShardsampProxies

DataPath

26

Data-PathandControlManagementPathSeparation

27

asadatagrid

MicroservicesArchitectureandPolyglotPersistence

28

AuthenticationAuthentication

Authentication

KeyValue

K V

KeyValue

K V

KeyValue

K V

Customers

KeyValue

K V

Graph

CustomersKeyValue

K V

Graph

Catalog

RDBMS Cache

Search

SessionStore

SessionStore

SessionStore

Document

Document

Document

API

API

API

API

API

API

API

API

API

FraudDetection

API

ColumnarSearch

Fulfilment API

RDBMS

TheCostofPolyglotPersistence

AuthenticationAuthentication

Authentication

KeyValue

K V

KeyValue

K V

KeyValue

K V

Customers

KeyValue

K V

Graph

CustomersKeyValue

K V

Graph

Catalog

RDBMS Cache

Search

SessionStore

SessionStore

SessionStore

Document

Document

Document

API

API

API

API

API

API

API

API

API

FraudDetection

API

ColumnarSearch

Fulfilment API

RDBMS

Increasedapplicationcomplexity CostlycommunicationApplicationdoesheavyliftinginsharingdatakeepingdatasetsinsync

HighoperationalburdenHighercostofownershipDifferentdatabaseshavespecializedadministrativescalingavailability

requirements

Sub-optimalResourceUsage HighercostDedicatingpodsserversforeachtypeofdatabasereducesdeploymentefficiency

API API API API APIAPI API

RedisEnterpriseAMulti-modelDatabaseforMicroservices

Authentication Customers Catalog Search FraudDetection SessionStore

SearchGraphKeyValue

RDBMS

KeyValueCache

RDBMS

Cache

Fulfilment

KeyValue Document

Authentication Customers Catalog Search FraudDetection SessionStore

SearchGraphKeyValue

RDBMS

KeyValueCache

RDBMS

Cache

Fulfilment

KeyValue Document

Built-inMessageBroker

Built-inPub-SubStreamsforeventsynchacrossdatastores

Whatarewemissing

bull Howtoconsumemessagesinthisldquobuiltinmessageborkerrdquo

bull GivenashardeddatabasehowcanIrunanalyticalqueries

bull MultiModeldatabasendash Singlecopyincoredatatypesndash Intermodulecommunicationndash ComponentXdoingtranslationsbetweenmodules

33

RedisGears

Introducing

RedisGearsisaServerlessengineformulti-modelandclusteroperationsinRedissupportingbotheventdriven

aswellasbatchoperations

WhatisRedisGears

GearsCoordinator MapReducerGearsExecuter

GearsinfrastructureiswritteninC

C-API

Soon Soon

HighPerformanceArchitecture

RedisGearsallowstodefineapipeofoperations

bull Returningvaluefromoneoperationispassedtothenextoperationinthepipe

bull Lastoperationreturningtheresulttotheuserbull Firstoperationiscalledlsquoreaderrsquo-responsibleforprovidingdata

ndash Keysreader-readkeysfromRedisndash Streamreader-readstreamsfromRedisndash Pythonreader-allowtousertowritehisownreadersinpython

ScriptingwithRedisGears

36

bull Map

bull FlatMap

bull Filterbull Groupby+Reducebull Aggregatebull Sortbull Limit

bull ForEachbull Distinct

SupportedOperations

37

Reader(Flat)Mapper

UsingRedisGearsndash(Flat)Mapping

UsingRedisGears-Filtering

Reader Filterrecordwith1doc

UsingRedisGears-Aggregate

Reader CountAggregator

1

1

Demo

GearshasastreamingAPItoallowtotriggergearsexecutiononeventsndash RedisStreamevents-Triggeranexecutionwheneveranewdataentersasteamndash RedisKeysevents-Triggeranoperationwheneverakeyistouched

UseCase1ndashStreamProcessing

RedisTimeSeriesRedisStreams

Everysec

BecauseofRedisGearsrsquoflexibility(itsactuallyrunningpython)youcanachieveinternalmoduleintegrationwithitndash ReadfromhashesandindexinRediSearchRedisGraphndash ReadRedisJSONdataandpasstoRedisTimeSeriesndash hellip

UseCase2ndashaMultiModelEngine

RediSearch

RedisHashesEveryupdate

RedisGraph

createthebuilderbuilder=GearsBuilder()filtereventsonkeyall_keysbuilderfilter(lambdaxx[key]=all_keys)addthekeystoall_keyssetbuildermap(lambdaxexecute(saddall_keysx[key]))registertheexecutiononkeyspacenotificationbuilderregister()

Buildagearthatcreatesmaintainsasetofallkeyswithinredis

Recipe1ndasheventriggering

createthepipebuilderKeysOnlyReaderisaperformanceimprovementonlypipingthekeysbuilder=GearsBuilder(KeysOnlyReader)getfromeachhashthegenresfieldbuildermap(lambdaxexecute(hgetxgenres))filterthosewhodonothavegenresbuilderfilter(lambdaxxisnotNone)splitgenresbycommabuilderflatmap(lambdaxxsplit())countforeachgenrethenumberoftimesitappearsbuildercountby()starttheexecutionbuilderrun(movie)

Buildagearthatcountshowoftenagenreisusedwithinasetofmovies

Recipe2ndashmapreducing

createthebuilderwithaStreamReaderbuilder=GearsBuilder(StreamReader)extracteachfieldvaluepairfromthemessageandincreasethepipegranularitybuilderflatmap(lambdax[(a[0]a[1])forainxitems()])filteroutthestreamIditselfbuilderfilter(lambdaxx[0]=streamId)makesurethegearsdatalivesinthecorrectshardbuilderrepartition(lambdaxx[0])applyeachfieldvaluepairtoakeybuilderforeach(lambdaxexecute(setx[0]x[1]))registeronnewmessagesonthestreaminputStreambuilderregister(inputStream)

Buildagearthatconsumesastreamandupdateskeysaccordingly

Recipe3ndashstreamprocessing

ExampleTriggerExplained

47

ExampleTriggerExplained

48

ExampleTriggerExplained

49

ExampleTriggerExplained-Flatmap

50

ExampleTriggerExplained-Repartition

51

ExampleTriggerExplained-executeCommand

52

Demo

DemoSetup

54

bull httpsgithubcomRedisGearsAnimalRecognitionDemo

bull redisfoundmycat

Challenge

55

RedisEnterpriseCluster

Node1 Node2 NodeN(oddnumber)

18

Unevennumberofsymmetricnodes

RedisEnterpriseCluster

Node1 Node2 NodeN(oddnumber)

19

Singlemasterdatabase

M

RedisEnterpriseCluster

Node1 Node2 NodeN(oddnumber)

20

AnHAdatabase

SM

RedisEnterpriseCluster

Node1 Node2 NodeN(oddnumber)

21

AClusteredDatabase

M1 M2 M3

Howdokeysgetassignedtopartitions

22

Howdokeysgetassignedtopartitions

23

RedisEnterpriseCluster

Node1 Node2 NodeN(oddnumber)

24

AHighlyAvailableClusteredDatabase

M1 M2 M3S3 S1 S2

RedisEnterpriseNode

25

ClusterManager

EnterpriseLayer

OpenSourceLayer

RESTAPIZerolatencyproxy

RedisShards

RedisEnterpriseSharedNothingSymmetricArchitecture

ClusterManagementPath

NodeWatchdogClusterWatchdog

Node1 Node2 NodeN(oddnumber)

RedisShardsampProxies

DataPath

26

Data-PathandControlManagementPathSeparation

27

asadatagrid

MicroservicesArchitectureandPolyglotPersistence

28

AuthenticationAuthentication

Authentication

KeyValue

K V

KeyValue

K V

KeyValue

K V

Customers

KeyValue

K V

Graph

CustomersKeyValue

K V

Graph

Catalog

RDBMS Cache

Search

SessionStore

SessionStore

SessionStore

Document

Document

Document

API

API

API

API

API

API

API

API

API

FraudDetection

API

ColumnarSearch

Fulfilment API

RDBMS

TheCostofPolyglotPersistence

AuthenticationAuthentication

Authentication

KeyValue

K V

KeyValue

K V

KeyValue

K V

Customers

KeyValue

K V

Graph

CustomersKeyValue

K V

Graph

Catalog

RDBMS Cache

Search

SessionStore

SessionStore

SessionStore

Document

Document

Document

API

API

API

API

API

API

API

API

API

FraudDetection

API

ColumnarSearch

Fulfilment API

RDBMS

Increasedapplicationcomplexity CostlycommunicationApplicationdoesheavyliftinginsharingdatakeepingdatasetsinsync

HighoperationalburdenHighercostofownershipDifferentdatabaseshavespecializedadministrativescalingavailability

requirements

Sub-optimalResourceUsage HighercostDedicatingpodsserversforeachtypeofdatabasereducesdeploymentefficiency

API API API API APIAPI API

RedisEnterpriseAMulti-modelDatabaseforMicroservices

Authentication Customers Catalog Search FraudDetection SessionStore

SearchGraphKeyValue

RDBMS

KeyValueCache

RDBMS

Cache

Fulfilment

KeyValue Document

Authentication Customers Catalog Search FraudDetection SessionStore

SearchGraphKeyValue

RDBMS

KeyValueCache

RDBMS

Cache

Fulfilment

KeyValue Document

Built-inMessageBroker

Built-inPub-SubStreamsforeventsynchacrossdatastores

Whatarewemissing

bull Howtoconsumemessagesinthisldquobuiltinmessageborkerrdquo

bull GivenashardeddatabasehowcanIrunanalyticalqueries

bull MultiModeldatabasendash Singlecopyincoredatatypesndash Intermodulecommunicationndash ComponentXdoingtranslationsbetweenmodules

33

RedisGears

Introducing

RedisGearsisaServerlessengineformulti-modelandclusteroperationsinRedissupportingbotheventdriven

aswellasbatchoperations

WhatisRedisGears

GearsCoordinator MapReducerGearsExecuter

GearsinfrastructureiswritteninC

C-API

Soon Soon

HighPerformanceArchitecture

RedisGearsallowstodefineapipeofoperations

bull Returningvaluefromoneoperationispassedtothenextoperationinthepipe

bull Lastoperationreturningtheresulttotheuserbull Firstoperationiscalledlsquoreaderrsquo-responsibleforprovidingdata

ndash Keysreader-readkeysfromRedisndash Streamreader-readstreamsfromRedisndash Pythonreader-allowtousertowritehisownreadersinpython

ScriptingwithRedisGears

36

bull Map

bull FlatMap

bull Filterbull Groupby+Reducebull Aggregatebull Sortbull Limit

bull ForEachbull Distinct

SupportedOperations

37

Reader(Flat)Mapper

UsingRedisGearsndash(Flat)Mapping

UsingRedisGears-Filtering

Reader Filterrecordwith1doc

UsingRedisGears-Aggregate

Reader CountAggregator

1

1

Demo

GearshasastreamingAPItoallowtotriggergearsexecutiononeventsndash RedisStreamevents-Triggeranexecutionwheneveranewdataentersasteamndash RedisKeysevents-Triggeranoperationwheneverakeyistouched

UseCase1ndashStreamProcessing

RedisTimeSeriesRedisStreams

Everysec

BecauseofRedisGearsrsquoflexibility(itsactuallyrunningpython)youcanachieveinternalmoduleintegrationwithitndash ReadfromhashesandindexinRediSearchRedisGraphndash ReadRedisJSONdataandpasstoRedisTimeSeriesndash hellip

UseCase2ndashaMultiModelEngine

RediSearch

RedisHashesEveryupdate

RedisGraph

createthebuilderbuilder=GearsBuilder()filtereventsonkeyall_keysbuilderfilter(lambdaxx[key]=all_keys)addthekeystoall_keyssetbuildermap(lambdaxexecute(saddall_keysx[key]))registertheexecutiononkeyspacenotificationbuilderregister()

Buildagearthatcreatesmaintainsasetofallkeyswithinredis

Recipe1ndasheventriggering

createthepipebuilderKeysOnlyReaderisaperformanceimprovementonlypipingthekeysbuilder=GearsBuilder(KeysOnlyReader)getfromeachhashthegenresfieldbuildermap(lambdaxexecute(hgetxgenres))filterthosewhodonothavegenresbuilderfilter(lambdaxxisnotNone)splitgenresbycommabuilderflatmap(lambdaxxsplit())countforeachgenrethenumberoftimesitappearsbuildercountby()starttheexecutionbuilderrun(movie)

Buildagearthatcountshowoftenagenreisusedwithinasetofmovies

Recipe2ndashmapreducing

createthebuilderwithaStreamReaderbuilder=GearsBuilder(StreamReader)extracteachfieldvaluepairfromthemessageandincreasethepipegranularitybuilderflatmap(lambdax[(a[0]a[1])forainxitems()])filteroutthestreamIditselfbuilderfilter(lambdaxx[0]=streamId)makesurethegearsdatalivesinthecorrectshardbuilderrepartition(lambdaxx[0])applyeachfieldvaluepairtoakeybuilderforeach(lambdaxexecute(setx[0]x[1]))registeronnewmessagesonthestreaminputStreambuilderregister(inputStream)

Buildagearthatconsumesastreamandupdateskeysaccordingly

Recipe3ndashstreamprocessing

ExampleTriggerExplained

47

ExampleTriggerExplained

48

ExampleTriggerExplained

49

ExampleTriggerExplained-Flatmap

50

ExampleTriggerExplained-Repartition

51

ExampleTriggerExplained-executeCommand

52

Demo

DemoSetup

54

bull httpsgithubcomRedisGearsAnimalRecognitionDemo

bull redisfoundmycat

Challenge

55

RedisEnterpriseCluster

Node1 Node2 NodeN(oddnumber)

19

Singlemasterdatabase

M

RedisEnterpriseCluster

Node1 Node2 NodeN(oddnumber)

20

AnHAdatabase

SM

RedisEnterpriseCluster

Node1 Node2 NodeN(oddnumber)

21

AClusteredDatabase

M1 M2 M3

Howdokeysgetassignedtopartitions

22

Howdokeysgetassignedtopartitions

23

RedisEnterpriseCluster

Node1 Node2 NodeN(oddnumber)

24

AHighlyAvailableClusteredDatabase

M1 M2 M3S3 S1 S2

RedisEnterpriseNode

25

ClusterManager

EnterpriseLayer

OpenSourceLayer

RESTAPIZerolatencyproxy

RedisShards

RedisEnterpriseSharedNothingSymmetricArchitecture

ClusterManagementPath

NodeWatchdogClusterWatchdog

Node1 Node2 NodeN(oddnumber)

RedisShardsampProxies

DataPath

26

Data-PathandControlManagementPathSeparation

27

asadatagrid

MicroservicesArchitectureandPolyglotPersistence

28

AuthenticationAuthentication

Authentication

KeyValue

K V

KeyValue

K V

KeyValue

K V

Customers

KeyValue

K V

Graph

CustomersKeyValue

K V

Graph

Catalog

RDBMS Cache

Search

SessionStore

SessionStore

SessionStore

Document

Document

Document

API

API

API

API

API

API

API

API

API

FraudDetection

API

ColumnarSearch

Fulfilment API

RDBMS

TheCostofPolyglotPersistence

AuthenticationAuthentication

Authentication

KeyValue

K V

KeyValue

K V

KeyValue

K V

Customers

KeyValue

K V

Graph

CustomersKeyValue

K V

Graph

Catalog

RDBMS Cache

Search

SessionStore

SessionStore

SessionStore

Document

Document

Document

API

API

API

API

API

API

API

API

API

FraudDetection

API

ColumnarSearch

Fulfilment API

RDBMS

Increasedapplicationcomplexity CostlycommunicationApplicationdoesheavyliftinginsharingdatakeepingdatasetsinsync

HighoperationalburdenHighercostofownershipDifferentdatabaseshavespecializedadministrativescalingavailability

requirements

Sub-optimalResourceUsage HighercostDedicatingpodsserversforeachtypeofdatabasereducesdeploymentefficiency

API API API API APIAPI API

RedisEnterpriseAMulti-modelDatabaseforMicroservices

Authentication Customers Catalog Search FraudDetection SessionStore

SearchGraphKeyValue

RDBMS

KeyValueCache

RDBMS

Cache

Fulfilment

KeyValue Document

Authentication Customers Catalog Search FraudDetection SessionStore

SearchGraphKeyValue

RDBMS

KeyValueCache

RDBMS

Cache

Fulfilment

KeyValue Document

Built-inMessageBroker

Built-inPub-SubStreamsforeventsynchacrossdatastores

Whatarewemissing

bull Howtoconsumemessagesinthisldquobuiltinmessageborkerrdquo

bull GivenashardeddatabasehowcanIrunanalyticalqueries

bull MultiModeldatabasendash Singlecopyincoredatatypesndash Intermodulecommunicationndash ComponentXdoingtranslationsbetweenmodules

33

RedisGears

Introducing

RedisGearsisaServerlessengineformulti-modelandclusteroperationsinRedissupportingbotheventdriven

aswellasbatchoperations

WhatisRedisGears

GearsCoordinator MapReducerGearsExecuter

GearsinfrastructureiswritteninC

C-API

Soon Soon

HighPerformanceArchitecture

RedisGearsallowstodefineapipeofoperations

bull Returningvaluefromoneoperationispassedtothenextoperationinthepipe

bull Lastoperationreturningtheresulttotheuserbull Firstoperationiscalledlsquoreaderrsquo-responsibleforprovidingdata

ndash Keysreader-readkeysfromRedisndash Streamreader-readstreamsfromRedisndash Pythonreader-allowtousertowritehisownreadersinpython

ScriptingwithRedisGears

36

bull Map

bull FlatMap

bull Filterbull Groupby+Reducebull Aggregatebull Sortbull Limit

bull ForEachbull Distinct

SupportedOperations

37

Reader(Flat)Mapper

UsingRedisGearsndash(Flat)Mapping

UsingRedisGears-Filtering

Reader Filterrecordwith1doc

UsingRedisGears-Aggregate

Reader CountAggregator

1

1

Demo

GearshasastreamingAPItoallowtotriggergearsexecutiononeventsndash RedisStreamevents-Triggeranexecutionwheneveranewdataentersasteamndash RedisKeysevents-Triggeranoperationwheneverakeyistouched

UseCase1ndashStreamProcessing

RedisTimeSeriesRedisStreams

Everysec

BecauseofRedisGearsrsquoflexibility(itsactuallyrunningpython)youcanachieveinternalmoduleintegrationwithitndash ReadfromhashesandindexinRediSearchRedisGraphndash ReadRedisJSONdataandpasstoRedisTimeSeriesndash hellip

UseCase2ndashaMultiModelEngine

RediSearch

RedisHashesEveryupdate

RedisGraph

createthebuilderbuilder=GearsBuilder()filtereventsonkeyall_keysbuilderfilter(lambdaxx[key]=all_keys)addthekeystoall_keyssetbuildermap(lambdaxexecute(saddall_keysx[key]))registertheexecutiononkeyspacenotificationbuilderregister()

Buildagearthatcreatesmaintainsasetofallkeyswithinredis

Recipe1ndasheventriggering

createthepipebuilderKeysOnlyReaderisaperformanceimprovementonlypipingthekeysbuilder=GearsBuilder(KeysOnlyReader)getfromeachhashthegenresfieldbuildermap(lambdaxexecute(hgetxgenres))filterthosewhodonothavegenresbuilderfilter(lambdaxxisnotNone)splitgenresbycommabuilderflatmap(lambdaxxsplit())countforeachgenrethenumberoftimesitappearsbuildercountby()starttheexecutionbuilderrun(movie)

Buildagearthatcountshowoftenagenreisusedwithinasetofmovies

Recipe2ndashmapreducing

createthebuilderwithaStreamReaderbuilder=GearsBuilder(StreamReader)extracteachfieldvaluepairfromthemessageandincreasethepipegranularitybuilderflatmap(lambdax[(a[0]a[1])forainxitems()])filteroutthestreamIditselfbuilderfilter(lambdaxx[0]=streamId)makesurethegearsdatalivesinthecorrectshardbuilderrepartition(lambdaxx[0])applyeachfieldvaluepairtoakeybuilderforeach(lambdaxexecute(setx[0]x[1]))registeronnewmessagesonthestreaminputStreambuilderregister(inputStream)

Buildagearthatconsumesastreamandupdateskeysaccordingly

Recipe3ndashstreamprocessing

ExampleTriggerExplained

47

ExampleTriggerExplained

48

ExampleTriggerExplained

49

ExampleTriggerExplained-Flatmap

50

ExampleTriggerExplained-Repartition

51

ExampleTriggerExplained-executeCommand

52

Demo

DemoSetup

54

bull httpsgithubcomRedisGearsAnimalRecognitionDemo

bull redisfoundmycat

Challenge

55

RedisEnterpriseCluster

Node1 Node2 NodeN(oddnumber)

20

AnHAdatabase

SM

RedisEnterpriseCluster

Node1 Node2 NodeN(oddnumber)

21

AClusteredDatabase

M1 M2 M3

Howdokeysgetassignedtopartitions

22

Howdokeysgetassignedtopartitions

23

RedisEnterpriseCluster

Node1 Node2 NodeN(oddnumber)

24

AHighlyAvailableClusteredDatabase

M1 M2 M3S3 S1 S2

RedisEnterpriseNode

25

ClusterManager

EnterpriseLayer

OpenSourceLayer

RESTAPIZerolatencyproxy

RedisShards

RedisEnterpriseSharedNothingSymmetricArchitecture

ClusterManagementPath

NodeWatchdogClusterWatchdog

Node1 Node2 NodeN(oddnumber)

RedisShardsampProxies

DataPath

26

Data-PathandControlManagementPathSeparation

27

asadatagrid

MicroservicesArchitectureandPolyglotPersistence

28

AuthenticationAuthentication

Authentication

KeyValue

K V

KeyValue

K V

KeyValue

K V

Customers

KeyValue

K V

Graph

CustomersKeyValue

K V

Graph

Catalog

RDBMS Cache

Search

SessionStore

SessionStore

SessionStore

Document

Document

Document

API

API

API

API

API

API

API

API

API

FraudDetection

API

ColumnarSearch

Fulfilment API

RDBMS

TheCostofPolyglotPersistence

AuthenticationAuthentication

Authentication

KeyValue

K V

KeyValue

K V

KeyValue

K V

Customers

KeyValue

K V

Graph

CustomersKeyValue

K V

Graph

Catalog

RDBMS Cache

Search

SessionStore

SessionStore

SessionStore

Document

Document

Document

API

API

API

API

API

API

API

API

API

FraudDetection

API

ColumnarSearch

Fulfilment API

RDBMS

Increasedapplicationcomplexity CostlycommunicationApplicationdoesheavyliftinginsharingdatakeepingdatasetsinsync

HighoperationalburdenHighercostofownershipDifferentdatabaseshavespecializedadministrativescalingavailability

requirements

Sub-optimalResourceUsage HighercostDedicatingpodsserversforeachtypeofdatabasereducesdeploymentefficiency

API API API API APIAPI API

RedisEnterpriseAMulti-modelDatabaseforMicroservices

Authentication Customers Catalog Search FraudDetection SessionStore

SearchGraphKeyValue

RDBMS

KeyValueCache

RDBMS

Cache

Fulfilment

KeyValue Document

Authentication Customers Catalog Search FraudDetection SessionStore

SearchGraphKeyValue

RDBMS

KeyValueCache

RDBMS

Cache

Fulfilment

KeyValue Document

Built-inMessageBroker

Built-inPub-SubStreamsforeventsynchacrossdatastores

Whatarewemissing

bull Howtoconsumemessagesinthisldquobuiltinmessageborkerrdquo

bull GivenashardeddatabasehowcanIrunanalyticalqueries

bull MultiModeldatabasendash Singlecopyincoredatatypesndash Intermodulecommunicationndash ComponentXdoingtranslationsbetweenmodules

33

RedisGears

Introducing

RedisGearsisaServerlessengineformulti-modelandclusteroperationsinRedissupportingbotheventdriven

aswellasbatchoperations

WhatisRedisGears

GearsCoordinator MapReducerGearsExecuter

GearsinfrastructureiswritteninC

C-API

Soon Soon

HighPerformanceArchitecture

RedisGearsallowstodefineapipeofoperations

bull Returningvaluefromoneoperationispassedtothenextoperationinthepipe

bull Lastoperationreturningtheresulttotheuserbull Firstoperationiscalledlsquoreaderrsquo-responsibleforprovidingdata

ndash Keysreader-readkeysfromRedisndash Streamreader-readstreamsfromRedisndash Pythonreader-allowtousertowritehisownreadersinpython

ScriptingwithRedisGears

36

bull Map

bull FlatMap

bull Filterbull Groupby+Reducebull Aggregatebull Sortbull Limit

bull ForEachbull Distinct

SupportedOperations

37

Reader(Flat)Mapper

UsingRedisGearsndash(Flat)Mapping

UsingRedisGears-Filtering

Reader Filterrecordwith1doc

UsingRedisGears-Aggregate

Reader CountAggregator

1

1

Demo

GearshasastreamingAPItoallowtotriggergearsexecutiononeventsndash RedisStreamevents-Triggeranexecutionwheneveranewdataentersasteamndash RedisKeysevents-Triggeranoperationwheneverakeyistouched

UseCase1ndashStreamProcessing

RedisTimeSeriesRedisStreams

Everysec

BecauseofRedisGearsrsquoflexibility(itsactuallyrunningpython)youcanachieveinternalmoduleintegrationwithitndash ReadfromhashesandindexinRediSearchRedisGraphndash ReadRedisJSONdataandpasstoRedisTimeSeriesndash hellip

UseCase2ndashaMultiModelEngine

RediSearch

RedisHashesEveryupdate

RedisGraph

createthebuilderbuilder=GearsBuilder()filtereventsonkeyall_keysbuilderfilter(lambdaxx[key]=all_keys)addthekeystoall_keyssetbuildermap(lambdaxexecute(saddall_keysx[key]))registertheexecutiononkeyspacenotificationbuilderregister()

Buildagearthatcreatesmaintainsasetofallkeyswithinredis

Recipe1ndasheventriggering

createthepipebuilderKeysOnlyReaderisaperformanceimprovementonlypipingthekeysbuilder=GearsBuilder(KeysOnlyReader)getfromeachhashthegenresfieldbuildermap(lambdaxexecute(hgetxgenres))filterthosewhodonothavegenresbuilderfilter(lambdaxxisnotNone)splitgenresbycommabuilderflatmap(lambdaxxsplit())countforeachgenrethenumberoftimesitappearsbuildercountby()starttheexecutionbuilderrun(movie)

Buildagearthatcountshowoftenagenreisusedwithinasetofmovies

Recipe2ndashmapreducing

createthebuilderwithaStreamReaderbuilder=GearsBuilder(StreamReader)extracteachfieldvaluepairfromthemessageandincreasethepipegranularitybuilderflatmap(lambdax[(a[0]a[1])forainxitems()])filteroutthestreamIditselfbuilderfilter(lambdaxx[0]=streamId)makesurethegearsdatalivesinthecorrectshardbuilderrepartition(lambdaxx[0])applyeachfieldvaluepairtoakeybuilderforeach(lambdaxexecute(setx[0]x[1]))registeronnewmessagesonthestreaminputStreambuilderregister(inputStream)

Buildagearthatconsumesastreamandupdateskeysaccordingly

Recipe3ndashstreamprocessing

ExampleTriggerExplained

47

ExampleTriggerExplained

48

ExampleTriggerExplained

49

ExampleTriggerExplained-Flatmap

50

ExampleTriggerExplained-Repartition

51

ExampleTriggerExplained-executeCommand

52

Demo

DemoSetup

54

bull httpsgithubcomRedisGearsAnimalRecognitionDemo

bull redisfoundmycat

Challenge

55

RedisEnterpriseCluster

Node1 Node2 NodeN(oddnumber)

21

AClusteredDatabase

M1 M2 M3

Howdokeysgetassignedtopartitions

22

Howdokeysgetassignedtopartitions

23

RedisEnterpriseCluster

Node1 Node2 NodeN(oddnumber)

24

AHighlyAvailableClusteredDatabase

M1 M2 M3S3 S1 S2

RedisEnterpriseNode

25

ClusterManager

EnterpriseLayer

OpenSourceLayer

RESTAPIZerolatencyproxy

RedisShards

RedisEnterpriseSharedNothingSymmetricArchitecture

ClusterManagementPath

NodeWatchdogClusterWatchdog

Node1 Node2 NodeN(oddnumber)

RedisShardsampProxies

DataPath

26

Data-PathandControlManagementPathSeparation

27

asadatagrid

MicroservicesArchitectureandPolyglotPersistence

28

AuthenticationAuthentication

Authentication

KeyValue

K V

KeyValue

K V

KeyValue

K V

Customers

KeyValue

K V

Graph

CustomersKeyValue

K V

Graph

Catalog

RDBMS Cache

Search

SessionStore

SessionStore

SessionStore

Document

Document

Document

API

API

API

API

API

API

API

API

API

FraudDetection

API

ColumnarSearch

Fulfilment API

RDBMS

TheCostofPolyglotPersistence

AuthenticationAuthentication

Authentication

KeyValue

K V

KeyValue

K V

KeyValue

K V

Customers

KeyValue

K V

Graph

CustomersKeyValue

K V

Graph

Catalog

RDBMS Cache

Search

SessionStore

SessionStore

SessionStore

Document

Document

Document

API

API

API

API

API

API

API

API

API

FraudDetection

API

ColumnarSearch

Fulfilment API

RDBMS

Increasedapplicationcomplexity CostlycommunicationApplicationdoesheavyliftinginsharingdatakeepingdatasetsinsync

HighoperationalburdenHighercostofownershipDifferentdatabaseshavespecializedadministrativescalingavailability

requirements

Sub-optimalResourceUsage HighercostDedicatingpodsserversforeachtypeofdatabasereducesdeploymentefficiency

API API API API APIAPI API

RedisEnterpriseAMulti-modelDatabaseforMicroservices

Authentication Customers Catalog Search FraudDetection SessionStore

SearchGraphKeyValue

RDBMS

KeyValueCache

RDBMS

Cache

Fulfilment

KeyValue Document

Authentication Customers Catalog Search FraudDetection SessionStore

SearchGraphKeyValue

RDBMS

KeyValueCache

RDBMS

Cache

Fulfilment

KeyValue Document

Built-inMessageBroker

Built-inPub-SubStreamsforeventsynchacrossdatastores

Whatarewemissing

bull Howtoconsumemessagesinthisldquobuiltinmessageborkerrdquo

bull GivenashardeddatabasehowcanIrunanalyticalqueries

bull MultiModeldatabasendash Singlecopyincoredatatypesndash Intermodulecommunicationndash ComponentXdoingtranslationsbetweenmodules

33

RedisGears

Introducing

RedisGearsisaServerlessengineformulti-modelandclusteroperationsinRedissupportingbotheventdriven

aswellasbatchoperations

WhatisRedisGears

GearsCoordinator MapReducerGearsExecuter

GearsinfrastructureiswritteninC

C-API

Soon Soon

HighPerformanceArchitecture

RedisGearsallowstodefineapipeofoperations

bull Returningvaluefromoneoperationispassedtothenextoperationinthepipe

bull Lastoperationreturningtheresulttotheuserbull Firstoperationiscalledlsquoreaderrsquo-responsibleforprovidingdata

ndash Keysreader-readkeysfromRedisndash Streamreader-readstreamsfromRedisndash Pythonreader-allowtousertowritehisownreadersinpython

ScriptingwithRedisGears

36

bull Map

bull FlatMap

bull Filterbull Groupby+Reducebull Aggregatebull Sortbull Limit

bull ForEachbull Distinct

SupportedOperations

37

Reader(Flat)Mapper

UsingRedisGearsndash(Flat)Mapping

UsingRedisGears-Filtering

Reader Filterrecordwith1doc

UsingRedisGears-Aggregate

Reader CountAggregator

1

1

Demo

GearshasastreamingAPItoallowtotriggergearsexecutiononeventsndash RedisStreamevents-Triggeranexecutionwheneveranewdataentersasteamndash RedisKeysevents-Triggeranoperationwheneverakeyistouched

UseCase1ndashStreamProcessing

RedisTimeSeriesRedisStreams

Everysec

BecauseofRedisGearsrsquoflexibility(itsactuallyrunningpython)youcanachieveinternalmoduleintegrationwithitndash ReadfromhashesandindexinRediSearchRedisGraphndash ReadRedisJSONdataandpasstoRedisTimeSeriesndash hellip

UseCase2ndashaMultiModelEngine

RediSearch

RedisHashesEveryupdate

RedisGraph

createthebuilderbuilder=GearsBuilder()filtereventsonkeyall_keysbuilderfilter(lambdaxx[key]=all_keys)addthekeystoall_keyssetbuildermap(lambdaxexecute(saddall_keysx[key]))registertheexecutiononkeyspacenotificationbuilderregister()

Buildagearthatcreatesmaintainsasetofallkeyswithinredis

Recipe1ndasheventriggering

createthepipebuilderKeysOnlyReaderisaperformanceimprovementonlypipingthekeysbuilder=GearsBuilder(KeysOnlyReader)getfromeachhashthegenresfieldbuildermap(lambdaxexecute(hgetxgenres))filterthosewhodonothavegenresbuilderfilter(lambdaxxisnotNone)splitgenresbycommabuilderflatmap(lambdaxxsplit())countforeachgenrethenumberoftimesitappearsbuildercountby()starttheexecutionbuilderrun(movie)

Buildagearthatcountshowoftenagenreisusedwithinasetofmovies

Recipe2ndashmapreducing

createthebuilderwithaStreamReaderbuilder=GearsBuilder(StreamReader)extracteachfieldvaluepairfromthemessageandincreasethepipegranularitybuilderflatmap(lambdax[(a[0]a[1])forainxitems()])filteroutthestreamIditselfbuilderfilter(lambdaxx[0]=streamId)makesurethegearsdatalivesinthecorrectshardbuilderrepartition(lambdaxx[0])applyeachfieldvaluepairtoakeybuilderforeach(lambdaxexecute(setx[0]x[1]))registeronnewmessagesonthestreaminputStreambuilderregister(inputStream)

Buildagearthatconsumesastreamandupdateskeysaccordingly

Recipe3ndashstreamprocessing

ExampleTriggerExplained

47

ExampleTriggerExplained

48

ExampleTriggerExplained

49

ExampleTriggerExplained-Flatmap

50

ExampleTriggerExplained-Repartition

51

ExampleTriggerExplained-executeCommand

52

Demo

DemoSetup

54

bull httpsgithubcomRedisGearsAnimalRecognitionDemo

bull redisfoundmycat

Challenge

55

Howdokeysgetassignedtopartitions

22

Howdokeysgetassignedtopartitions

23

RedisEnterpriseCluster

Node1 Node2 NodeN(oddnumber)

24

AHighlyAvailableClusteredDatabase

M1 M2 M3S3 S1 S2

RedisEnterpriseNode

25

ClusterManager

EnterpriseLayer

OpenSourceLayer

RESTAPIZerolatencyproxy

RedisShards

RedisEnterpriseSharedNothingSymmetricArchitecture

ClusterManagementPath

NodeWatchdogClusterWatchdog

Node1 Node2 NodeN(oddnumber)

RedisShardsampProxies

DataPath

26

Data-PathandControlManagementPathSeparation

27

asadatagrid

MicroservicesArchitectureandPolyglotPersistence

28

AuthenticationAuthentication

Authentication

KeyValue

K V

KeyValue

K V

KeyValue

K V

Customers

KeyValue

K V

Graph

CustomersKeyValue

K V

Graph

Catalog

RDBMS Cache

Search

SessionStore

SessionStore

SessionStore

Document

Document

Document

API

API

API

API

API

API

API

API

API

FraudDetection

API

ColumnarSearch

Fulfilment API

RDBMS

TheCostofPolyglotPersistence

AuthenticationAuthentication

Authentication

KeyValue

K V

KeyValue

K V

KeyValue

K V

Customers

KeyValue

K V

Graph

CustomersKeyValue

K V

Graph

Catalog

RDBMS Cache

Search

SessionStore

SessionStore

SessionStore

Document

Document

Document

API

API

API

API

API

API

API

API

API

FraudDetection

API

ColumnarSearch

Fulfilment API

RDBMS

Increasedapplicationcomplexity CostlycommunicationApplicationdoesheavyliftinginsharingdatakeepingdatasetsinsync

HighoperationalburdenHighercostofownershipDifferentdatabaseshavespecializedadministrativescalingavailability

requirements

Sub-optimalResourceUsage HighercostDedicatingpodsserversforeachtypeofdatabasereducesdeploymentefficiency

API API API API APIAPI API

RedisEnterpriseAMulti-modelDatabaseforMicroservices

Authentication Customers Catalog Search FraudDetection SessionStore

SearchGraphKeyValue

RDBMS

KeyValueCache

RDBMS

Cache

Fulfilment

KeyValue Document

Authentication Customers Catalog Search FraudDetection SessionStore

SearchGraphKeyValue

RDBMS

KeyValueCache

RDBMS

Cache

Fulfilment

KeyValue Document

Built-inMessageBroker

Built-inPub-SubStreamsforeventsynchacrossdatastores

Whatarewemissing

bull Howtoconsumemessagesinthisldquobuiltinmessageborkerrdquo

bull GivenashardeddatabasehowcanIrunanalyticalqueries

bull MultiModeldatabasendash Singlecopyincoredatatypesndash Intermodulecommunicationndash ComponentXdoingtranslationsbetweenmodules

33

RedisGears

Introducing

RedisGearsisaServerlessengineformulti-modelandclusteroperationsinRedissupportingbotheventdriven

aswellasbatchoperations

WhatisRedisGears

GearsCoordinator MapReducerGearsExecuter

GearsinfrastructureiswritteninC

C-API

Soon Soon

HighPerformanceArchitecture

RedisGearsallowstodefineapipeofoperations

bull Returningvaluefromoneoperationispassedtothenextoperationinthepipe

bull Lastoperationreturningtheresulttotheuserbull Firstoperationiscalledlsquoreaderrsquo-responsibleforprovidingdata

ndash Keysreader-readkeysfromRedisndash Streamreader-readstreamsfromRedisndash Pythonreader-allowtousertowritehisownreadersinpython

ScriptingwithRedisGears

36

bull Map

bull FlatMap

bull Filterbull Groupby+Reducebull Aggregatebull Sortbull Limit

bull ForEachbull Distinct

SupportedOperations

37

Reader(Flat)Mapper

UsingRedisGearsndash(Flat)Mapping

UsingRedisGears-Filtering

Reader Filterrecordwith1doc

UsingRedisGears-Aggregate

Reader CountAggregator

1

1

Demo

GearshasastreamingAPItoallowtotriggergearsexecutiononeventsndash RedisStreamevents-Triggeranexecutionwheneveranewdataentersasteamndash RedisKeysevents-Triggeranoperationwheneverakeyistouched

UseCase1ndashStreamProcessing

RedisTimeSeriesRedisStreams

Everysec

BecauseofRedisGearsrsquoflexibility(itsactuallyrunningpython)youcanachieveinternalmoduleintegrationwithitndash ReadfromhashesandindexinRediSearchRedisGraphndash ReadRedisJSONdataandpasstoRedisTimeSeriesndash hellip

UseCase2ndashaMultiModelEngine

RediSearch

RedisHashesEveryupdate

RedisGraph

createthebuilderbuilder=GearsBuilder()filtereventsonkeyall_keysbuilderfilter(lambdaxx[key]=all_keys)addthekeystoall_keyssetbuildermap(lambdaxexecute(saddall_keysx[key]))registertheexecutiononkeyspacenotificationbuilderregister()

Buildagearthatcreatesmaintainsasetofallkeyswithinredis

Recipe1ndasheventriggering

createthepipebuilderKeysOnlyReaderisaperformanceimprovementonlypipingthekeysbuilder=GearsBuilder(KeysOnlyReader)getfromeachhashthegenresfieldbuildermap(lambdaxexecute(hgetxgenres))filterthosewhodonothavegenresbuilderfilter(lambdaxxisnotNone)splitgenresbycommabuilderflatmap(lambdaxxsplit())countforeachgenrethenumberoftimesitappearsbuildercountby()starttheexecutionbuilderrun(movie)

Buildagearthatcountshowoftenagenreisusedwithinasetofmovies

Recipe2ndashmapreducing

createthebuilderwithaStreamReaderbuilder=GearsBuilder(StreamReader)extracteachfieldvaluepairfromthemessageandincreasethepipegranularitybuilderflatmap(lambdax[(a[0]a[1])forainxitems()])filteroutthestreamIditselfbuilderfilter(lambdaxx[0]=streamId)makesurethegearsdatalivesinthecorrectshardbuilderrepartition(lambdaxx[0])applyeachfieldvaluepairtoakeybuilderforeach(lambdaxexecute(setx[0]x[1]))registeronnewmessagesonthestreaminputStreambuilderregister(inputStream)

Buildagearthatconsumesastreamandupdateskeysaccordingly

Recipe3ndashstreamprocessing

ExampleTriggerExplained

47

ExampleTriggerExplained

48

ExampleTriggerExplained

49

ExampleTriggerExplained-Flatmap

50

ExampleTriggerExplained-Repartition

51

ExampleTriggerExplained-executeCommand

52

Demo

DemoSetup

54

bull httpsgithubcomRedisGearsAnimalRecognitionDemo

bull redisfoundmycat

Challenge

55

Howdokeysgetassignedtopartitions

23

RedisEnterpriseCluster

Node1 Node2 NodeN(oddnumber)

24

AHighlyAvailableClusteredDatabase

M1 M2 M3S3 S1 S2

RedisEnterpriseNode

25

ClusterManager

EnterpriseLayer

OpenSourceLayer

RESTAPIZerolatencyproxy

RedisShards

RedisEnterpriseSharedNothingSymmetricArchitecture

ClusterManagementPath

NodeWatchdogClusterWatchdog

Node1 Node2 NodeN(oddnumber)

RedisShardsampProxies

DataPath

26

Data-PathandControlManagementPathSeparation

27

asadatagrid

MicroservicesArchitectureandPolyglotPersistence

28

AuthenticationAuthentication

Authentication

KeyValue

K V

KeyValue

K V

KeyValue

K V

Customers

KeyValue

K V

Graph

CustomersKeyValue

K V

Graph

Catalog

RDBMS Cache

Search

SessionStore

SessionStore

SessionStore

Document

Document

Document

API

API

API

API

API

API

API

API

API

FraudDetection

API

ColumnarSearch

Fulfilment API

RDBMS

TheCostofPolyglotPersistence

AuthenticationAuthentication

Authentication

KeyValue

K V

KeyValue

K V

KeyValue

K V

Customers

KeyValue

K V

Graph

CustomersKeyValue

K V

Graph

Catalog

RDBMS Cache

Search

SessionStore

SessionStore

SessionStore

Document

Document

Document

API

API

API

API

API

API

API

API

API

FraudDetection

API

ColumnarSearch

Fulfilment API

RDBMS

Increasedapplicationcomplexity CostlycommunicationApplicationdoesheavyliftinginsharingdatakeepingdatasetsinsync

HighoperationalburdenHighercostofownershipDifferentdatabaseshavespecializedadministrativescalingavailability

requirements

Sub-optimalResourceUsage HighercostDedicatingpodsserversforeachtypeofdatabasereducesdeploymentefficiency

API API API API APIAPI API

RedisEnterpriseAMulti-modelDatabaseforMicroservices

Authentication Customers Catalog Search FraudDetection SessionStore

SearchGraphKeyValue

RDBMS

KeyValueCache

RDBMS

Cache

Fulfilment

KeyValue Document

Authentication Customers Catalog Search FraudDetection SessionStore

SearchGraphKeyValue

RDBMS

KeyValueCache

RDBMS

Cache

Fulfilment

KeyValue Document

Built-inMessageBroker

Built-inPub-SubStreamsforeventsynchacrossdatastores

Whatarewemissing

bull Howtoconsumemessagesinthisldquobuiltinmessageborkerrdquo

bull GivenashardeddatabasehowcanIrunanalyticalqueries

bull MultiModeldatabasendash Singlecopyincoredatatypesndash Intermodulecommunicationndash ComponentXdoingtranslationsbetweenmodules

33

RedisGears

Introducing

RedisGearsisaServerlessengineformulti-modelandclusteroperationsinRedissupportingbotheventdriven

aswellasbatchoperations

WhatisRedisGears

GearsCoordinator MapReducerGearsExecuter

GearsinfrastructureiswritteninC

C-API

Soon Soon

HighPerformanceArchitecture

RedisGearsallowstodefineapipeofoperations

bull Returningvaluefromoneoperationispassedtothenextoperationinthepipe

bull Lastoperationreturningtheresulttotheuserbull Firstoperationiscalledlsquoreaderrsquo-responsibleforprovidingdata

ndash Keysreader-readkeysfromRedisndash Streamreader-readstreamsfromRedisndash Pythonreader-allowtousertowritehisownreadersinpython

ScriptingwithRedisGears

36

bull Map

bull FlatMap

bull Filterbull Groupby+Reducebull Aggregatebull Sortbull Limit

bull ForEachbull Distinct

SupportedOperations

37

Reader(Flat)Mapper

UsingRedisGearsndash(Flat)Mapping

UsingRedisGears-Filtering

Reader Filterrecordwith1doc

UsingRedisGears-Aggregate

Reader CountAggregator

1

1

Demo

GearshasastreamingAPItoallowtotriggergearsexecutiononeventsndash RedisStreamevents-Triggeranexecutionwheneveranewdataentersasteamndash RedisKeysevents-Triggeranoperationwheneverakeyistouched

UseCase1ndashStreamProcessing

RedisTimeSeriesRedisStreams

Everysec

BecauseofRedisGearsrsquoflexibility(itsactuallyrunningpython)youcanachieveinternalmoduleintegrationwithitndash ReadfromhashesandindexinRediSearchRedisGraphndash ReadRedisJSONdataandpasstoRedisTimeSeriesndash hellip

UseCase2ndashaMultiModelEngine

RediSearch

RedisHashesEveryupdate

RedisGraph

createthebuilderbuilder=GearsBuilder()filtereventsonkeyall_keysbuilderfilter(lambdaxx[key]=all_keys)addthekeystoall_keyssetbuildermap(lambdaxexecute(saddall_keysx[key]))registertheexecutiononkeyspacenotificationbuilderregister()

Buildagearthatcreatesmaintainsasetofallkeyswithinredis

Recipe1ndasheventriggering

createthepipebuilderKeysOnlyReaderisaperformanceimprovementonlypipingthekeysbuilder=GearsBuilder(KeysOnlyReader)getfromeachhashthegenresfieldbuildermap(lambdaxexecute(hgetxgenres))filterthosewhodonothavegenresbuilderfilter(lambdaxxisnotNone)splitgenresbycommabuilderflatmap(lambdaxxsplit())countforeachgenrethenumberoftimesitappearsbuildercountby()starttheexecutionbuilderrun(movie)

Buildagearthatcountshowoftenagenreisusedwithinasetofmovies

Recipe2ndashmapreducing

createthebuilderwithaStreamReaderbuilder=GearsBuilder(StreamReader)extracteachfieldvaluepairfromthemessageandincreasethepipegranularitybuilderflatmap(lambdax[(a[0]a[1])forainxitems()])filteroutthestreamIditselfbuilderfilter(lambdaxx[0]=streamId)makesurethegearsdatalivesinthecorrectshardbuilderrepartition(lambdaxx[0])applyeachfieldvaluepairtoakeybuilderforeach(lambdaxexecute(setx[0]x[1]))registeronnewmessagesonthestreaminputStreambuilderregister(inputStream)

Buildagearthatconsumesastreamandupdateskeysaccordingly

Recipe3ndashstreamprocessing

ExampleTriggerExplained

47

ExampleTriggerExplained

48

ExampleTriggerExplained

49

ExampleTriggerExplained-Flatmap

50

ExampleTriggerExplained-Repartition

51

ExampleTriggerExplained-executeCommand

52

Demo

DemoSetup

54

bull httpsgithubcomRedisGearsAnimalRecognitionDemo

bull redisfoundmycat

Challenge

55

RedisEnterpriseCluster

Node1 Node2 NodeN(oddnumber)

24

AHighlyAvailableClusteredDatabase

M1 M2 M3S3 S1 S2

RedisEnterpriseNode

25

ClusterManager

EnterpriseLayer

OpenSourceLayer

RESTAPIZerolatencyproxy

RedisShards

RedisEnterpriseSharedNothingSymmetricArchitecture

ClusterManagementPath

NodeWatchdogClusterWatchdog

Node1 Node2 NodeN(oddnumber)

RedisShardsampProxies

DataPath

26

Data-PathandControlManagementPathSeparation

27

asadatagrid

MicroservicesArchitectureandPolyglotPersistence

28

AuthenticationAuthentication

Authentication

KeyValue

K V

KeyValue

K V

KeyValue

K V

Customers

KeyValue

K V

Graph

CustomersKeyValue

K V

Graph

Catalog

RDBMS Cache

Search

SessionStore

SessionStore

SessionStore

Document

Document

Document

API

API

API

API

API

API

API

API

API

FraudDetection

API

ColumnarSearch

Fulfilment API

RDBMS

TheCostofPolyglotPersistence

AuthenticationAuthentication

Authentication

KeyValue

K V

KeyValue

K V

KeyValue

K V

Customers

KeyValue

K V

Graph

CustomersKeyValue

K V

Graph

Catalog

RDBMS Cache

Search

SessionStore

SessionStore

SessionStore

Document

Document

Document

API

API

API

API

API

API

API

API

API

FraudDetection

API

ColumnarSearch

Fulfilment API

RDBMS

Increasedapplicationcomplexity CostlycommunicationApplicationdoesheavyliftinginsharingdatakeepingdatasetsinsync

HighoperationalburdenHighercostofownershipDifferentdatabaseshavespecializedadministrativescalingavailability

requirements

Sub-optimalResourceUsage HighercostDedicatingpodsserversforeachtypeofdatabasereducesdeploymentefficiency

API API API API APIAPI API

RedisEnterpriseAMulti-modelDatabaseforMicroservices

Authentication Customers Catalog Search FraudDetection SessionStore

SearchGraphKeyValue

RDBMS

KeyValueCache

RDBMS

Cache

Fulfilment

KeyValue Document

Authentication Customers Catalog Search FraudDetection SessionStore

SearchGraphKeyValue

RDBMS

KeyValueCache

RDBMS

Cache

Fulfilment

KeyValue Document

Built-inMessageBroker

Built-inPub-SubStreamsforeventsynchacrossdatastores

Whatarewemissing

bull Howtoconsumemessagesinthisldquobuiltinmessageborkerrdquo

bull GivenashardeddatabasehowcanIrunanalyticalqueries

bull MultiModeldatabasendash Singlecopyincoredatatypesndash Intermodulecommunicationndash ComponentXdoingtranslationsbetweenmodules

33

RedisGears

Introducing

RedisGearsisaServerlessengineformulti-modelandclusteroperationsinRedissupportingbotheventdriven

aswellasbatchoperations

WhatisRedisGears

GearsCoordinator MapReducerGearsExecuter

GearsinfrastructureiswritteninC

C-API

Soon Soon

HighPerformanceArchitecture

RedisGearsallowstodefineapipeofoperations

bull Returningvaluefromoneoperationispassedtothenextoperationinthepipe

bull Lastoperationreturningtheresulttotheuserbull Firstoperationiscalledlsquoreaderrsquo-responsibleforprovidingdata

ndash Keysreader-readkeysfromRedisndash Streamreader-readstreamsfromRedisndash Pythonreader-allowtousertowritehisownreadersinpython

ScriptingwithRedisGears

36

bull Map

bull FlatMap

bull Filterbull Groupby+Reducebull Aggregatebull Sortbull Limit

bull ForEachbull Distinct

SupportedOperations

37

Reader(Flat)Mapper

UsingRedisGearsndash(Flat)Mapping

UsingRedisGears-Filtering

Reader Filterrecordwith1doc

UsingRedisGears-Aggregate

Reader CountAggregator

1

1

Demo

GearshasastreamingAPItoallowtotriggergearsexecutiononeventsndash RedisStreamevents-Triggeranexecutionwheneveranewdataentersasteamndash RedisKeysevents-Triggeranoperationwheneverakeyistouched

UseCase1ndashStreamProcessing

RedisTimeSeriesRedisStreams

Everysec

BecauseofRedisGearsrsquoflexibility(itsactuallyrunningpython)youcanachieveinternalmoduleintegrationwithitndash ReadfromhashesandindexinRediSearchRedisGraphndash ReadRedisJSONdataandpasstoRedisTimeSeriesndash hellip

UseCase2ndashaMultiModelEngine

RediSearch

RedisHashesEveryupdate

RedisGraph

createthebuilderbuilder=GearsBuilder()filtereventsonkeyall_keysbuilderfilter(lambdaxx[key]=all_keys)addthekeystoall_keyssetbuildermap(lambdaxexecute(saddall_keysx[key]))registertheexecutiononkeyspacenotificationbuilderregister()

Buildagearthatcreatesmaintainsasetofallkeyswithinredis

Recipe1ndasheventriggering

createthepipebuilderKeysOnlyReaderisaperformanceimprovementonlypipingthekeysbuilder=GearsBuilder(KeysOnlyReader)getfromeachhashthegenresfieldbuildermap(lambdaxexecute(hgetxgenres))filterthosewhodonothavegenresbuilderfilter(lambdaxxisnotNone)splitgenresbycommabuilderflatmap(lambdaxxsplit())countforeachgenrethenumberoftimesitappearsbuildercountby()starttheexecutionbuilderrun(movie)

Buildagearthatcountshowoftenagenreisusedwithinasetofmovies

Recipe2ndashmapreducing

createthebuilderwithaStreamReaderbuilder=GearsBuilder(StreamReader)extracteachfieldvaluepairfromthemessageandincreasethepipegranularitybuilderflatmap(lambdax[(a[0]a[1])forainxitems()])filteroutthestreamIditselfbuilderfilter(lambdaxx[0]=streamId)makesurethegearsdatalivesinthecorrectshardbuilderrepartition(lambdaxx[0])applyeachfieldvaluepairtoakeybuilderforeach(lambdaxexecute(setx[0]x[1]))registeronnewmessagesonthestreaminputStreambuilderregister(inputStream)

Buildagearthatconsumesastreamandupdateskeysaccordingly

Recipe3ndashstreamprocessing

ExampleTriggerExplained

47

ExampleTriggerExplained

48

ExampleTriggerExplained

49

ExampleTriggerExplained-Flatmap

50

ExampleTriggerExplained-Repartition

51

ExampleTriggerExplained-executeCommand

52

Demo

DemoSetup

54

bull httpsgithubcomRedisGearsAnimalRecognitionDemo

bull redisfoundmycat

Challenge

55

RedisEnterpriseNode

25

ClusterManager

EnterpriseLayer

OpenSourceLayer

RESTAPIZerolatencyproxy

RedisShards

RedisEnterpriseSharedNothingSymmetricArchitecture

ClusterManagementPath

NodeWatchdogClusterWatchdog

Node1 Node2 NodeN(oddnumber)

RedisShardsampProxies

DataPath

26

Data-PathandControlManagementPathSeparation

27

asadatagrid

MicroservicesArchitectureandPolyglotPersistence

28

AuthenticationAuthentication

Authentication

KeyValue

K V

KeyValue

K V

KeyValue

K V

Customers

KeyValue

K V

Graph

CustomersKeyValue

K V

Graph

Catalog

RDBMS Cache

Search

SessionStore

SessionStore

SessionStore

Document

Document

Document

API

API

API

API

API

API

API

API

API

FraudDetection

API

ColumnarSearch

Fulfilment API

RDBMS

TheCostofPolyglotPersistence

AuthenticationAuthentication

Authentication

KeyValue

K V

KeyValue

K V

KeyValue

K V

Customers

KeyValue

K V

Graph

CustomersKeyValue

K V

Graph

Catalog

RDBMS Cache

Search

SessionStore

SessionStore

SessionStore

Document

Document

Document

API

API

API

API

API

API

API

API

API

FraudDetection

API

ColumnarSearch

Fulfilment API

RDBMS

Increasedapplicationcomplexity CostlycommunicationApplicationdoesheavyliftinginsharingdatakeepingdatasetsinsync

HighoperationalburdenHighercostofownershipDifferentdatabaseshavespecializedadministrativescalingavailability

requirements

Sub-optimalResourceUsage HighercostDedicatingpodsserversforeachtypeofdatabasereducesdeploymentefficiency

API API API API APIAPI API

RedisEnterpriseAMulti-modelDatabaseforMicroservices

Authentication Customers Catalog Search FraudDetection SessionStore

SearchGraphKeyValue

RDBMS

KeyValueCache

RDBMS

Cache

Fulfilment

KeyValue Document

Authentication Customers Catalog Search FraudDetection SessionStore

SearchGraphKeyValue

RDBMS

KeyValueCache

RDBMS

Cache

Fulfilment

KeyValue Document

Built-inMessageBroker

Built-inPub-SubStreamsforeventsynchacrossdatastores

Whatarewemissing

bull Howtoconsumemessagesinthisldquobuiltinmessageborkerrdquo

bull GivenashardeddatabasehowcanIrunanalyticalqueries

bull MultiModeldatabasendash Singlecopyincoredatatypesndash Intermodulecommunicationndash ComponentXdoingtranslationsbetweenmodules

33

RedisGears

Introducing

RedisGearsisaServerlessengineformulti-modelandclusteroperationsinRedissupportingbotheventdriven

aswellasbatchoperations

WhatisRedisGears

GearsCoordinator MapReducerGearsExecuter

GearsinfrastructureiswritteninC

C-API

Soon Soon

HighPerformanceArchitecture

RedisGearsallowstodefineapipeofoperations

bull Returningvaluefromoneoperationispassedtothenextoperationinthepipe

bull Lastoperationreturningtheresulttotheuserbull Firstoperationiscalledlsquoreaderrsquo-responsibleforprovidingdata

ndash Keysreader-readkeysfromRedisndash Streamreader-readstreamsfromRedisndash Pythonreader-allowtousertowritehisownreadersinpython

ScriptingwithRedisGears

36

bull Map

bull FlatMap

bull Filterbull Groupby+Reducebull Aggregatebull Sortbull Limit

bull ForEachbull Distinct

SupportedOperations

37

Reader(Flat)Mapper

UsingRedisGearsndash(Flat)Mapping

UsingRedisGears-Filtering

Reader Filterrecordwith1doc

UsingRedisGears-Aggregate

Reader CountAggregator

1

1

Demo

GearshasastreamingAPItoallowtotriggergearsexecutiononeventsndash RedisStreamevents-Triggeranexecutionwheneveranewdataentersasteamndash RedisKeysevents-Triggeranoperationwheneverakeyistouched

UseCase1ndashStreamProcessing

RedisTimeSeriesRedisStreams

Everysec

BecauseofRedisGearsrsquoflexibility(itsactuallyrunningpython)youcanachieveinternalmoduleintegrationwithitndash ReadfromhashesandindexinRediSearchRedisGraphndash ReadRedisJSONdataandpasstoRedisTimeSeriesndash hellip

UseCase2ndashaMultiModelEngine

RediSearch

RedisHashesEveryupdate

RedisGraph

createthebuilderbuilder=GearsBuilder()filtereventsonkeyall_keysbuilderfilter(lambdaxx[key]=all_keys)addthekeystoall_keyssetbuildermap(lambdaxexecute(saddall_keysx[key]))registertheexecutiononkeyspacenotificationbuilderregister()

Buildagearthatcreatesmaintainsasetofallkeyswithinredis

Recipe1ndasheventriggering

createthepipebuilderKeysOnlyReaderisaperformanceimprovementonlypipingthekeysbuilder=GearsBuilder(KeysOnlyReader)getfromeachhashthegenresfieldbuildermap(lambdaxexecute(hgetxgenres))filterthosewhodonothavegenresbuilderfilter(lambdaxxisnotNone)splitgenresbycommabuilderflatmap(lambdaxxsplit())countforeachgenrethenumberoftimesitappearsbuildercountby()starttheexecutionbuilderrun(movie)

Buildagearthatcountshowoftenagenreisusedwithinasetofmovies

Recipe2ndashmapreducing

createthebuilderwithaStreamReaderbuilder=GearsBuilder(StreamReader)extracteachfieldvaluepairfromthemessageandincreasethepipegranularitybuilderflatmap(lambdax[(a[0]a[1])forainxitems()])filteroutthestreamIditselfbuilderfilter(lambdaxx[0]=streamId)makesurethegearsdatalivesinthecorrectshardbuilderrepartition(lambdaxx[0])applyeachfieldvaluepairtoakeybuilderforeach(lambdaxexecute(setx[0]x[1]))registeronnewmessagesonthestreaminputStreambuilderregister(inputStream)

Buildagearthatconsumesastreamandupdateskeysaccordingly

Recipe3ndashstreamprocessing

ExampleTriggerExplained

47

ExampleTriggerExplained

48

ExampleTriggerExplained

49

ExampleTriggerExplained-Flatmap

50

ExampleTriggerExplained-Repartition

51

ExampleTriggerExplained-executeCommand

52

Demo

DemoSetup

54

bull httpsgithubcomRedisGearsAnimalRecognitionDemo

bull redisfoundmycat

Challenge

55

RedisEnterpriseSharedNothingSymmetricArchitecture

ClusterManagementPath

NodeWatchdogClusterWatchdog

Node1 Node2 NodeN(oddnumber)

RedisShardsampProxies

DataPath

26

Data-PathandControlManagementPathSeparation

27

asadatagrid

MicroservicesArchitectureandPolyglotPersistence

28

AuthenticationAuthentication

Authentication

KeyValue

K V

KeyValue

K V

KeyValue

K V

Customers

KeyValue

K V

Graph

CustomersKeyValue

K V

Graph

Catalog

RDBMS Cache

Search

SessionStore

SessionStore

SessionStore

Document

Document

Document

API

API

API

API

API

API

API

API

API

FraudDetection

API

ColumnarSearch

Fulfilment API

RDBMS

TheCostofPolyglotPersistence

AuthenticationAuthentication

Authentication

KeyValue

K V

KeyValue

K V

KeyValue

K V

Customers

KeyValue

K V

Graph

CustomersKeyValue

K V

Graph

Catalog

RDBMS Cache

Search

SessionStore

SessionStore

SessionStore

Document

Document

Document

API

API

API

API

API

API

API

API

API

FraudDetection

API

ColumnarSearch

Fulfilment API

RDBMS

Increasedapplicationcomplexity CostlycommunicationApplicationdoesheavyliftinginsharingdatakeepingdatasetsinsync

HighoperationalburdenHighercostofownershipDifferentdatabaseshavespecializedadministrativescalingavailability

requirements

Sub-optimalResourceUsage HighercostDedicatingpodsserversforeachtypeofdatabasereducesdeploymentefficiency

API API API API APIAPI API

RedisEnterpriseAMulti-modelDatabaseforMicroservices

Authentication Customers Catalog Search FraudDetection SessionStore

SearchGraphKeyValue

RDBMS

KeyValueCache

RDBMS

Cache

Fulfilment

KeyValue Document

Authentication Customers Catalog Search FraudDetection SessionStore

SearchGraphKeyValue

RDBMS

KeyValueCache

RDBMS

Cache

Fulfilment

KeyValue Document

Built-inMessageBroker

Built-inPub-SubStreamsforeventsynchacrossdatastores

Whatarewemissing

bull Howtoconsumemessagesinthisldquobuiltinmessageborkerrdquo

bull GivenashardeddatabasehowcanIrunanalyticalqueries

bull MultiModeldatabasendash Singlecopyincoredatatypesndash Intermodulecommunicationndash ComponentXdoingtranslationsbetweenmodules

33

RedisGears

Introducing

RedisGearsisaServerlessengineformulti-modelandclusteroperationsinRedissupportingbotheventdriven

aswellasbatchoperations

WhatisRedisGears

GearsCoordinator MapReducerGearsExecuter

GearsinfrastructureiswritteninC

C-API

Soon Soon

HighPerformanceArchitecture

RedisGearsallowstodefineapipeofoperations

bull Returningvaluefromoneoperationispassedtothenextoperationinthepipe

bull Lastoperationreturningtheresulttotheuserbull Firstoperationiscalledlsquoreaderrsquo-responsibleforprovidingdata

ndash Keysreader-readkeysfromRedisndash Streamreader-readstreamsfromRedisndash Pythonreader-allowtousertowritehisownreadersinpython

ScriptingwithRedisGears

36

bull Map

bull FlatMap

bull Filterbull Groupby+Reducebull Aggregatebull Sortbull Limit

bull ForEachbull Distinct

SupportedOperations

37

Reader(Flat)Mapper

UsingRedisGearsndash(Flat)Mapping

UsingRedisGears-Filtering

Reader Filterrecordwith1doc

UsingRedisGears-Aggregate

Reader CountAggregator

1

1

Demo

GearshasastreamingAPItoallowtotriggergearsexecutiononeventsndash RedisStreamevents-Triggeranexecutionwheneveranewdataentersasteamndash RedisKeysevents-Triggeranoperationwheneverakeyistouched

UseCase1ndashStreamProcessing

RedisTimeSeriesRedisStreams

Everysec

BecauseofRedisGearsrsquoflexibility(itsactuallyrunningpython)youcanachieveinternalmoduleintegrationwithitndash ReadfromhashesandindexinRediSearchRedisGraphndash ReadRedisJSONdataandpasstoRedisTimeSeriesndash hellip

UseCase2ndashaMultiModelEngine

RediSearch

RedisHashesEveryupdate

RedisGraph

createthebuilderbuilder=GearsBuilder()filtereventsonkeyall_keysbuilderfilter(lambdaxx[key]=all_keys)addthekeystoall_keyssetbuildermap(lambdaxexecute(saddall_keysx[key]))registertheexecutiononkeyspacenotificationbuilderregister()

Buildagearthatcreatesmaintainsasetofallkeyswithinredis

Recipe1ndasheventriggering

createthepipebuilderKeysOnlyReaderisaperformanceimprovementonlypipingthekeysbuilder=GearsBuilder(KeysOnlyReader)getfromeachhashthegenresfieldbuildermap(lambdaxexecute(hgetxgenres))filterthosewhodonothavegenresbuilderfilter(lambdaxxisnotNone)splitgenresbycommabuilderflatmap(lambdaxxsplit())countforeachgenrethenumberoftimesitappearsbuildercountby()starttheexecutionbuilderrun(movie)

Buildagearthatcountshowoftenagenreisusedwithinasetofmovies

Recipe2ndashmapreducing

createthebuilderwithaStreamReaderbuilder=GearsBuilder(StreamReader)extracteachfieldvaluepairfromthemessageandincreasethepipegranularitybuilderflatmap(lambdax[(a[0]a[1])forainxitems()])filteroutthestreamIditselfbuilderfilter(lambdaxx[0]=streamId)makesurethegearsdatalivesinthecorrectshardbuilderrepartition(lambdaxx[0])applyeachfieldvaluepairtoakeybuilderforeach(lambdaxexecute(setx[0]x[1]))registeronnewmessagesonthestreaminputStreambuilderregister(inputStream)

Buildagearthatconsumesastreamandupdateskeysaccordingly

Recipe3ndashstreamprocessing

ExampleTriggerExplained

47

ExampleTriggerExplained

48

ExampleTriggerExplained

49

ExampleTriggerExplained-Flatmap

50

ExampleTriggerExplained-Repartition

51

ExampleTriggerExplained-executeCommand

52

Demo

DemoSetup

54

bull httpsgithubcomRedisGearsAnimalRecognitionDemo

bull redisfoundmycat

Challenge

55

27

asadatagrid

MicroservicesArchitectureandPolyglotPersistence

28

AuthenticationAuthentication

Authentication

KeyValue

K V

KeyValue

K V

KeyValue

K V

Customers

KeyValue

K V

Graph

CustomersKeyValue

K V

Graph

Catalog

RDBMS Cache

Search

SessionStore

SessionStore

SessionStore

Document

Document

Document

API

API

API

API

API

API

API

API

API

FraudDetection

API

ColumnarSearch

Fulfilment API

RDBMS

TheCostofPolyglotPersistence

AuthenticationAuthentication

Authentication

KeyValue

K V

KeyValue

K V

KeyValue

K V

Customers

KeyValue

K V

Graph

CustomersKeyValue

K V

Graph

Catalog

RDBMS Cache

Search

SessionStore

SessionStore

SessionStore

Document

Document

Document

API

API

API

API

API

API

API

API

API

FraudDetection

API

ColumnarSearch

Fulfilment API

RDBMS

Increasedapplicationcomplexity CostlycommunicationApplicationdoesheavyliftinginsharingdatakeepingdatasetsinsync

HighoperationalburdenHighercostofownershipDifferentdatabaseshavespecializedadministrativescalingavailability

requirements

Sub-optimalResourceUsage HighercostDedicatingpodsserversforeachtypeofdatabasereducesdeploymentefficiency

API API API API APIAPI API

RedisEnterpriseAMulti-modelDatabaseforMicroservices

Authentication Customers Catalog Search FraudDetection SessionStore

SearchGraphKeyValue

RDBMS

KeyValueCache

RDBMS

Cache

Fulfilment

KeyValue Document

Authentication Customers Catalog Search FraudDetection SessionStore

SearchGraphKeyValue

RDBMS

KeyValueCache

RDBMS

Cache

Fulfilment

KeyValue Document

Built-inMessageBroker

Built-inPub-SubStreamsforeventsynchacrossdatastores

Whatarewemissing

bull Howtoconsumemessagesinthisldquobuiltinmessageborkerrdquo

bull GivenashardeddatabasehowcanIrunanalyticalqueries

bull MultiModeldatabasendash Singlecopyincoredatatypesndash Intermodulecommunicationndash ComponentXdoingtranslationsbetweenmodules

33

RedisGears

Introducing

RedisGearsisaServerlessengineformulti-modelandclusteroperationsinRedissupportingbotheventdriven

aswellasbatchoperations

WhatisRedisGears

GearsCoordinator MapReducerGearsExecuter

GearsinfrastructureiswritteninC

C-API

Soon Soon

HighPerformanceArchitecture

RedisGearsallowstodefineapipeofoperations

bull Returningvaluefromoneoperationispassedtothenextoperationinthepipe

bull Lastoperationreturningtheresulttotheuserbull Firstoperationiscalledlsquoreaderrsquo-responsibleforprovidingdata

ndash Keysreader-readkeysfromRedisndash Streamreader-readstreamsfromRedisndash Pythonreader-allowtousertowritehisownreadersinpython

ScriptingwithRedisGears

36

bull Map

bull FlatMap

bull Filterbull Groupby+Reducebull Aggregatebull Sortbull Limit

bull ForEachbull Distinct

SupportedOperations

37

Reader(Flat)Mapper

UsingRedisGearsndash(Flat)Mapping

UsingRedisGears-Filtering

Reader Filterrecordwith1doc

UsingRedisGears-Aggregate

Reader CountAggregator

1

1

Demo

GearshasastreamingAPItoallowtotriggergearsexecutiononeventsndash RedisStreamevents-Triggeranexecutionwheneveranewdataentersasteamndash RedisKeysevents-Triggeranoperationwheneverakeyistouched

UseCase1ndashStreamProcessing

RedisTimeSeriesRedisStreams

Everysec

BecauseofRedisGearsrsquoflexibility(itsactuallyrunningpython)youcanachieveinternalmoduleintegrationwithitndash ReadfromhashesandindexinRediSearchRedisGraphndash ReadRedisJSONdataandpasstoRedisTimeSeriesndash hellip

UseCase2ndashaMultiModelEngine

RediSearch

RedisHashesEveryupdate

RedisGraph

createthebuilderbuilder=GearsBuilder()filtereventsonkeyall_keysbuilderfilter(lambdaxx[key]=all_keys)addthekeystoall_keyssetbuildermap(lambdaxexecute(saddall_keysx[key]))registertheexecutiononkeyspacenotificationbuilderregister()

Buildagearthatcreatesmaintainsasetofallkeyswithinredis

Recipe1ndasheventriggering

createthepipebuilderKeysOnlyReaderisaperformanceimprovementonlypipingthekeysbuilder=GearsBuilder(KeysOnlyReader)getfromeachhashthegenresfieldbuildermap(lambdaxexecute(hgetxgenres))filterthosewhodonothavegenresbuilderfilter(lambdaxxisnotNone)splitgenresbycommabuilderflatmap(lambdaxxsplit())countforeachgenrethenumberoftimesitappearsbuildercountby()starttheexecutionbuilderrun(movie)

Buildagearthatcountshowoftenagenreisusedwithinasetofmovies

Recipe2ndashmapreducing

createthebuilderwithaStreamReaderbuilder=GearsBuilder(StreamReader)extracteachfieldvaluepairfromthemessageandincreasethepipegranularitybuilderflatmap(lambdax[(a[0]a[1])forainxitems()])filteroutthestreamIditselfbuilderfilter(lambdaxx[0]=streamId)makesurethegearsdatalivesinthecorrectshardbuilderrepartition(lambdaxx[0])applyeachfieldvaluepairtoakeybuilderforeach(lambdaxexecute(setx[0]x[1]))registeronnewmessagesonthestreaminputStreambuilderregister(inputStream)

Buildagearthatconsumesastreamandupdateskeysaccordingly

Recipe3ndashstreamprocessing

ExampleTriggerExplained

47

ExampleTriggerExplained

48

ExampleTriggerExplained

49

ExampleTriggerExplained-Flatmap

50

ExampleTriggerExplained-Repartition

51

ExampleTriggerExplained-executeCommand

52

Demo

DemoSetup

54

bull httpsgithubcomRedisGearsAnimalRecognitionDemo

bull redisfoundmycat

Challenge

55

MicroservicesArchitectureandPolyglotPersistence

28

AuthenticationAuthentication

Authentication

KeyValue

K V

KeyValue

K V

KeyValue

K V

Customers

KeyValue

K V

Graph

CustomersKeyValue

K V

Graph

Catalog

RDBMS Cache

Search

SessionStore

SessionStore

SessionStore

Document

Document

Document

API

API

API

API

API

API

API

API

API

FraudDetection

API

ColumnarSearch

Fulfilment API

RDBMS

TheCostofPolyglotPersistence

AuthenticationAuthentication

Authentication

KeyValue

K V

KeyValue

K V

KeyValue

K V

Customers

KeyValue

K V

Graph

CustomersKeyValue

K V

Graph

Catalog

RDBMS Cache

Search

SessionStore

SessionStore

SessionStore

Document

Document

Document

API

API

API

API

API

API

API

API

API

FraudDetection

API

ColumnarSearch

Fulfilment API

RDBMS

Increasedapplicationcomplexity CostlycommunicationApplicationdoesheavyliftinginsharingdatakeepingdatasetsinsync

HighoperationalburdenHighercostofownershipDifferentdatabaseshavespecializedadministrativescalingavailability

requirements

Sub-optimalResourceUsage HighercostDedicatingpodsserversforeachtypeofdatabasereducesdeploymentefficiency

API API API API APIAPI API

RedisEnterpriseAMulti-modelDatabaseforMicroservices

Authentication Customers Catalog Search FraudDetection SessionStore

SearchGraphKeyValue

RDBMS

KeyValueCache

RDBMS

Cache

Fulfilment

KeyValue Document

Authentication Customers Catalog Search FraudDetection SessionStore

SearchGraphKeyValue

RDBMS

KeyValueCache

RDBMS

Cache

Fulfilment

KeyValue Document

Built-inMessageBroker

Built-inPub-SubStreamsforeventsynchacrossdatastores

Whatarewemissing

bull Howtoconsumemessagesinthisldquobuiltinmessageborkerrdquo

bull GivenashardeddatabasehowcanIrunanalyticalqueries

bull MultiModeldatabasendash Singlecopyincoredatatypesndash Intermodulecommunicationndash ComponentXdoingtranslationsbetweenmodules

33

RedisGears

Introducing

RedisGearsisaServerlessengineformulti-modelandclusteroperationsinRedissupportingbotheventdriven

aswellasbatchoperations

WhatisRedisGears

GearsCoordinator MapReducerGearsExecuter

GearsinfrastructureiswritteninC

C-API

Soon Soon

HighPerformanceArchitecture

RedisGearsallowstodefineapipeofoperations

bull Returningvaluefromoneoperationispassedtothenextoperationinthepipe

bull Lastoperationreturningtheresulttotheuserbull Firstoperationiscalledlsquoreaderrsquo-responsibleforprovidingdata

ndash Keysreader-readkeysfromRedisndash Streamreader-readstreamsfromRedisndash Pythonreader-allowtousertowritehisownreadersinpython

ScriptingwithRedisGears

36

bull Map

bull FlatMap

bull Filterbull Groupby+Reducebull Aggregatebull Sortbull Limit

bull ForEachbull Distinct

SupportedOperations

37

Reader(Flat)Mapper

UsingRedisGearsndash(Flat)Mapping

UsingRedisGears-Filtering

Reader Filterrecordwith1doc

UsingRedisGears-Aggregate

Reader CountAggregator

1

1

Demo

GearshasastreamingAPItoallowtotriggergearsexecutiononeventsndash RedisStreamevents-Triggeranexecutionwheneveranewdataentersasteamndash RedisKeysevents-Triggeranoperationwheneverakeyistouched

UseCase1ndashStreamProcessing

RedisTimeSeriesRedisStreams

Everysec

BecauseofRedisGearsrsquoflexibility(itsactuallyrunningpython)youcanachieveinternalmoduleintegrationwithitndash ReadfromhashesandindexinRediSearchRedisGraphndash ReadRedisJSONdataandpasstoRedisTimeSeriesndash hellip

UseCase2ndashaMultiModelEngine

RediSearch

RedisHashesEveryupdate

RedisGraph

createthebuilderbuilder=GearsBuilder()filtereventsonkeyall_keysbuilderfilter(lambdaxx[key]=all_keys)addthekeystoall_keyssetbuildermap(lambdaxexecute(saddall_keysx[key]))registertheexecutiononkeyspacenotificationbuilderregister()

Buildagearthatcreatesmaintainsasetofallkeyswithinredis

Recipe1ndasheventriggering

createthepipebuilderKeysOnlyReaderisaperformanceimprovementonlypipingthekeysbuilder=GearsBuilder(KeysOnlyReader)getfromeachhashthegenresfieldbuildermap(lambdaxexecute(hgetxgenres))filterthosewhodonothavegenresbuilderfilter(lambdaxxisnotNone)splitgenresbycommabuilderflatmap(lambdaxxsplit())countforeachgenrethenumberoftimesitappearsbuildercountby()starttheexecutionbuilderrun(movie)

Buildagearthatcountshowoftenagenreisusedwithinasetofmovies

Recipe2ndashmapreducing

createthebuilderwithaStreamReaderbuilder=GearsBuilder(StreamReader)extracteachfieldvaluepairfromthemessageandincreasethepipegranularitybuilderflatmap(lambdax[(a[0]a[1])forainxitems()])filteroutthestreamIditselfbuilderfilter(lambdaxx[0]=streamId)makesurethegearsdatalivesinthecorrectshardbuilderrepartition(lambdaxx[0])applyeachfieldvaluepairtoakeybuilderforeach(lambdaxexecute(setx[0]x[1]))registeronnewmessagesonthestreaminputStreambuilderregister(inputStream)

Buildagearthatconsumesastreamandupdateskeysaccordingly

Recipe3ndashstreamprocessing

ExampleTriggerExplained

47

ExampleTriggerExplained

48

ExampleTriggerExplained

49

ExampleTriggerExplained-Flatmap

50

ExampleTriggerExplained-Repartition

51

ExampleTriggerExplained-executeCommand

52

Demo

DemoSetup

54

bull httpsgithubcomRedisGearsAnimalRecognitionDemo

bull redisfoundmycat

Challenge

55

TheCostofPolyglotPersistence

AuthenticationAuthentication

Authentication

KeyValue

K V

KeyValue

K V

KeyValue

K V

Customers

KeyValue

K V

Graph

CustomersKeyValue

K V

Graph

Catalog

RDBMS Cache

Search

SessionStore

SessionStore

SessionStore

Document

Document

Document

API

API

API

API

API

API

API

API

API

FraudDetection

API

ColumnarSearch

Fulfilment API

RDBMS

Increasedapplicationcomplexity CostlycommunicationApplicationdoesheavyliftinginsharingdatakeepingdatasetsinsync

HighoperationalburdenHighercostofownershipDifferentdatabaseshavespecializedadministrativescalingavailability

requirements

Sub-optimalResourceUsage HighercostDedicatingpodsserversforeachtypeofdatabasereducesdeploymentefficiency

API API API API APIAPI API

RedisEnterpriseAMulti-modelDatabaseforMicroservices

Authentication Customers Catalog Search FraudDetection SessionStore

SearchGraphKeyValue

RDBMS

KeyValueCache

RDBMS

Cache

Fulfilment

KeyValue Document

Authentication Customers Catalog Search FraudDetection SessionStore

SearchGraphKeyValue

RDBMS

KeyValueCache

RDBMS

Cache

Fulfilment

KeyValue Document

Built-inMessageBroker

Built-inPub-SubStreamsforeventsynchacrossdatastores

Whatarewemissing

bull Howtoconsumemessagesinthisldquobuiltinmessageborkerrdquo

bull GivenashardeddatabasehowcanIrunanalyticalqueries

bull MultiModeldatabasendash Singlecopyincoredatatypesndash Intermodulecommunicationndash ComponentXdoingtranslationsbetweenmodules

33

RedisGears

Introducing

RedisGearsisaServerlessengineformulti-modelandclusteroperationsinRedissupportingbotheventdriven

aswellasbatchoperations

WhatisRedisGears

GearsCoordinator MapReducerGearsExecuter

GearsinfrastructureiswritteninC

C-API

Soon Soon

HighPerformanceArchitecture

RedisGearsallowstodefineapipeofoperations

bull Returningvaluefromoneoperationispassedtothenextoperationinthepipe

bull Lastoperationreturningtheresulttotheuserbull Firstoperationiscalledlsquoreaderrsquo-responsibleforprovidingdata

ndash Keysreader-readkeysfromRedisndash Streamreader-readstreamsfromRedisndash Pythonreader-allowtousertowritehisownreadersinpython

ScriptingwithRedisGears

36

bull Map

bull FlatMap

bull Filterbull Groupby+Reducebull Aggregatebull Sortbull Limit

bull ForEachbull Distinct

SupportedOperations

37

Reader(Flat)Mapper

UsingRedisGearsndash(Flat)Mapping

UsingRedisGears-Filtering

Reader Filterrecordwith1doc

UsingRedisGears-Aggregate

Reader CountAggregator

1

1

Demo

GearshasastreamingAPItoallowtotriggergearsexecutiononeventsndash RedisStreamevents-Triggeranexecutionwheneveranewdataentersasteamndash RedisKeysevents-Triggeranoperationwheneverakeyistouched

UseCase1ndashStreamProcessing

RedisTimeSeriesRedisStreams

Everysec

BecauseofRedisGearsrsquoflexibility(itsactuallyrunningpython)youcanachieveinternalmoduleintegrationwithitndash ReadfromhashesandindexinRediSearchRedisGraphndash ReadRedisJSONdataandpasstoRedisTimeSeriesndash hellip

UseCase2ndashaMultiModelEngine

RediSearch

RedisHashesEveryupdate

RedisGraph

createthebuilderbuilder=GearsBuilder()filtereventsonkeyall_keysbuilderfilter(lambdaxx[key]=all_keys)addthekeystoall_keyssetbuildermap(lambdaxexecute(saddall_keysx[key]))registertheexecutiononkeyspacenotificationbuilderregister()

Buildagearthatcreatesmaintainsasetofallkeyswithinredis

Recipe1ndasheventriggering

createthepipebuilderKeysOnlyReaderisaperformanceimprovementonlypipingthekeysbuilder=GearsBuilder(KeysOnlyReader)getfromeachhashthegenresfieldbuildermap(lambdaxexecute(hgetxgenres))filterthosewhodonothavegenresbuilderfilter(lambdaxxisnotNone)splitgenresbycommabuilderflatmap(lambdaxxsplit())countforeachgenrethenumberoftimesitappearsbuildercountby()starttheexecutionbuilderrun(movie)

Buildagearthatcountshowoftenagenreisusedwithinasetofmovies

Recipe2ndashmapreducing

createthebuilderwithaStreamReaderbuilder=GearsBuilder(StreamReader)extracteachfieldvaluepairfromthemessageandincreasethepipegranularitybuilderflatmap(lambdax[(a[0]a[1])forainxitems()])filteroutthestreamIditselfbuilderfilter(lambdaxx[0]=streamId)makesurethegearsdatalivesinthecorrectshardbuilderrepartition(lambdaxx[0])applyeachfieldvaluepairtoakeybuilderforeach(lambdaxexecute(setx[0]x[1]))registeronnewmessagesonthestreaminputStreambuilderregister(inputStream)

Buildagearthatconsumesastreamandupdateskeysaccordingly

Recipe3ndashstreamprocessing

ExampleTriggerExplained

47

ExampleTriggerExplained

48

ExampleTriggerExplained

49

ExampleTriggerExplained-Flatmap

50

ExampleTriggerExplained-Repartition

51

ExampleTriggerExplained-executeCommand

52

Demo

DemoSetup

54

bull httpsgithubcomRedisGearsAnimalRecognitionDemo

bull redisfoundmycat

Challenge

55

API API API API APIAPI API

RedisEnterpriseAMulti-modelDatabaseforMicroservices

Authentication Customers Catalog Search FraudDetection SessionStore

SearchGraphKeyValue

RDBMS

KeyValueCache

RDBMS

Cache

Fulfilment

KeyValue Document

Authentication Customers Catalog Search FraudDetection SessionStore

SearchGraphKeyValue

RDBMS

KeyValueCache

RDBMS

Cache

Fulfilment

KeyValue Document

Built-inMessageBroker

Built-inPub-SubStreamsforeventsynchacrossdatastores

Whatarewemissing

bull Howtoconsumemessagesinthisldquobuiltinmessageborkerrdquo

bull GivenashardeddatabasehowcanIrunanalyticalqueries

bull MultiModeldatabasendash Singlecopyincoredatatypesndash Intermodulecommunicationndash ComponentXdoingtranslationsbetweenmodules

33

RedisGears

Introducing

RedisGearsisaServerlessengineformulti-modelandclusteroperationsinRedissupportingbotheventdriven

aswellasbatchoperations

WhatisRedisGears

GearsCoordinator MapReducerGearsExecuter

GearsinfrastructureiswritteninC

C-API

Soon Soon

HighPerformanceArchitecture

RedisGearsallowstodefineapipeofoperations

bull Returningvaluefromoneoperationispassedtothenextoperationinthepipe

bull Lastoperationreturningtheresulttotheuserbull Firstoperationiscalledlsquoreaderrsquo-responsibleforprovidingdata

ndash Keysreader-readkeysfromRedisndash Streamreader-readstreamsfromRedisndash Pythonreader-allowtousertowritehisownreadersinpython

ScriptingwithRedisGears

36

bull Map

bull FlatMap

bull Filterbull Groupby+Reducebull Aggregatebull Sortbull Limit

bull ForEachbull Distinct

SupportedOperations

37

Reader(Flat)Mapper

UsingRedisGearsndash(Flat)Mapping

UsingRedisGears-Filtering

Reader Filterrecordwith1doc

UsingRedisGears-Aggregate

Reader CountAggregator

1

1

Demo

GearshasastreamingAPItoallowtotriggergearsexecutiononeventsndash RedisStreamevents-Triggeranexecutionwheneveranewdataentersasteamndash RedisKeysevents-Triggeranoperationwheneverakeyistouched

UseCase1ndashStreamProcessing

RedisTimeSeriesRedisStreams

Everysec

BecauseofRedisGearsrsquoflexibility(itsactuallyrunningpython)youcanachieveinternalmoduleintegrationwithitndash ReadfromhashesandindexinRediSearchRedisGraphndash ReadRedisJSONdataandpasstoRedisTimeSeriesndash hellip

UseCase2ndashaMultiModelEngine

RediSearch

RedisHashesEveryupdate

RedisGraph

createthebuilderbuilder=GearsBuilder()filtereventsonkeyall_keysbuilderfilter(lambdaxx[key]=all_keys)addthekeystoall_keyssetbuildermap(lambdaxexecute(saddall_keysx[key]))registertheexecutiononkeyspacenotificationbuilderregister()

Buildagearthatcreatesmaintainsasetofallkeyswithinredis

Recipe1ndasheventriggering

createthepipebuilderKeysOnlyReaderisaperformanceimprovementonlypipingthekeysbuilder=GearsBuilder(KeysOnlyReader)getfromeachhashthegenresfieldbuildermap(lambdaxexecute(hgetxgenres))filterthosewhodonothavegenresbuilderfilter(lambdaxxisnotNone)splitgenresbycommabuilderflatmap(lambdaxxsplit())countforeachgenrethenumberoftimesitappearsbuildercountby()starttheexecutionbuilderrun(movie)

Buildagearthatcountshowoftenagenreisusedwithinasetofmovies

Recipe2ndashmapreducing

createthebuilderwithaStreamReaderbuilder=GearsBuilder(StreamReader)extracteachfieldvaluepairfromthemessageandincreasethepipegranularitybuilderflatmap(lambdax[(a[0]a[1])forainxitems()])filteroutthestreamIditselfbuilderfilter(lambdaxx[0]=streamId)makesurethegearsdatalivesinthecorrectshardbuilderrepartition(lambdaxx[0])applyeachfieldvaluepairtoakeybuilderforeach(lambdaxexecute(setx[0]x[1]))registeronnewmessagesonthestreaminputStreambuilderregister(inputStream)

Buildagearthatconsumesastreamandupdateskeysaccordingly

Recipe3ndashstreamprocessing

ExampleTriggerExplained

47

ExampleTriggerExplained

48

ExampleTriggerExplained

49

ExampleTriggerExplained-Flatmap

50

ExampleTriggerExplained-Repartition

51

ExampleTriggerExplained-executeCommand

52

Demo

DemoSetup

54

bull httpsgithubcomRedisGearsAnimalRecognitionDemo

bull redisfoundmycat

Challenge

55

Authentication Customers Catalog Search FraudDetection SessionStore

SearchGraphKeyValue

RDBMS

KeyValueCache

RDBMS

Cache

Fulfilment

KeyValue Document

Built-inMessageBroker

Built-inPub-SubStreamsforeventsynchacrossdatastores

Whatarewemissing

bull Howtoconsumemessagesinthisldquobuiltinmessageborkerrdquo

bull GivenashardeddatabasehowcanIrunanalyticalqueries

bull MultiModeldatabasendash Singlecopyincoredatatypesndash Intermodulecommunicationndash ComponentXdoingtranslationsbetweenmodules

33

RedisGears

Introducing

RedisGearsisaServerlessengineformulti-modelandclusteroperationsinRedissupportingbotheventdriven

aswellasbatchoperations

WhatisRedisGears

GearsCoordinator MapReducerGearsExecuter

GearsinfrastructureiswritteninC

C-API

Soon Soon

HighPerformanceArchitecture

RedisGearsallowstodefineapipeofoperations

bull Returningvaluefromoneoperationispassedtothenextoperationinthepipe

bull Lastoperationreturningtheresulttotheuserbull Firstoperationiscalledlsquoreaderrsquo-responsibleforprovidingdata

ndash Keysreader-readkeysfromRedisndash Streamreader-readstreamsfromRedisndash Pythonreader-allowtousertowritehisownreadersinpython

ScriptingwithRedisGears

36

bull Map

bull FlatMap

bull Filterbull Groupby+Reducebull Aggregatebull Sortbull Limit

bull ForEachbull Distinct

SupportedOperations

37

Reader(Flat)Mapper

UsingRedisGearsndash(Flat)Mapping

UsingRedisGears-Filtering

Reader Filterrecordwith1doc

UsingRedisGears-Aggregate

Reader CountAggregator

1

1

Demo

GearshasastreamingAPItoallowtotriggergearsexecutiononeventsndash RedisStreamevents-Triggeranexecutionwheneveranewdataentersasteamndash RedisKeysevents-Triggeranoperationwheneverakeyistouched

UseCase1ndashStreamProcessing

RedisTimeSeriesRedisStreams

Everysec

BecauseofRedisGearsrsquoflexibility(itsactuallyrunningpython)youcanachieveinternalmoduleintegrationwithitndash ReadfromhashesandindexinRediSearchRedisGraphndash ReadRedisJSONdataandpasstoRedisTimeSeriesndash hellip

UseCase2ndashaMultiModelEngine

RediSearch

RedisHashesEveryupdate

RedisGraph

createthebuilderbuilder=GearsBuilder()filtereventsonkeyall_keysbuilderfilter(lambdaxx[key]=all_keys)addthekeystoall_keyssetbuildermap(lambdaxexecute(saddall_keysx[key]))registertheexecutiononkeyspacenotificationbuilderregister()

Buildagearthatcreatesmaintainsasetofallkeyswithinredis

Recipe1ndasheventriggering

createthepipebuilderKeysOnlyReaderisaperformanceimprovementonlypipingthekeysbuilder=GearsBuilder(KeysOnlyReader)getfromeachhashthegenresfieldbuildermap(lambdaxexecute(hgetxgenres))filterthosewhodonothavegenresbuilderfilter(lambdaxxisnotNone)splitgenresbycommabuilderflatmap(lambdaxxsplit())countforeachgenrethenumberoftimesitappearsbuildercountby()starttheexecutionbuilderrun(movie)

Buildagearthatcountshowoftenagenreisusedwithinasetofmovies

Recipe2ndashmapreducing

createthebuilderwithaStreamReaderbuilder=GearsBuilder(StreamReader)extracteachfieldvaluepairfromthemessageandincreasethepipegranularitybuilderflatmap(lambdax[(a[0]a[1])forainxitems()])filteroutthestreamIditselfbuilderfilter(lambdaxx[0]=streamId)makesurethegearsdatalivesinthecorrectshardbuilderrepartition(lambdaxx[0])applyeachfieldvaluepairtoakeybuilderforeach(lambdaxexecute(setx[0]x[1]))registeronnewmessagesonthestreaminputStreambuilderregister(inputStream)

Buildagearthatconsumesastreamandupdateskeysaccordingly

Recipe3ndashstreamprocessing

ExampleTriggerExplained

47

ExampleTriggerExplained

48

ExampleTriggerExplained

49

ExampleTriggerExplained-Flatmap

50

ExampleTriggerExplained-Repartition

51

ExampleTriggerExplained-executeCommand

52

Demo

DemoSetup

54

bull httpsgithubcomRedisGearsAnimalRecognitionDemo

bull redisfoundmycat

Challenge

55

Whatarewemissing

bull Howtoconsumemessagesinthisldquobuiltinmessageborkerrdquo

bull GivenashardeddatabasehowcanIrunanalyticalqueries

bull MultiModeldatabasendash Singlecopyincoredatatypesndash Intermodulecommunicationndash ComponentXdoingtranslationsbetweenmodules

33

RedisGears

Introducing

RedisGearsisaServerlessengineformulti-modelandclusteroperationsinRedissupportingbotheventdriven

aswellasbatchoperations

WhatisRedisGears

GearsCoordinator MapReducerGearsExecuter

GearsinfrastructureiswritteninC

C-API

Soon Soon

HighPerformanceArchitecture

RedisGearsallowstodefineapipeofoperations

bull Returningvaluefromoneoperationispassedtothenextoperationinthepipe

bull Lastoperationreturningtheresulttotheuserbull Firstoperationiscalledlsquoreaderrsquo-responsibleforprovidingdata

ndash Keysreader-readkeysfromRedisndash Streamreader-readstreamsfromRedisndash Pythonreader-allowtousertowritehisownreadersinpython

ScriptingwithRedisGears

36

bull Map

bull FlatMap

bull Filterbull Groupby+Reducebull Aggregatebull Sortbull Limit

bull ForEachbull Distinct

SupportedOperations

37

Reader(Flat)Mapper

UsingRedisGearsndash(Flat)Mapping

UsingRedisGears-Filtering

Reader Filterrecordwith1doc

UsingRedisGears-Aggregate

Reader CountAggregator

1

1

Demo

GearshasastreamingAPItoallowtotriggergearsexecutiononeventsndash RedisStreamevents-Triggeranexecutionwheneveranewdataentersasteamndash RedisKeysevents-Triggeranoperationwheneverakeyistouched

UseCase1ndashStreamProcessing

RedisTimeSeriesRedisStreams

Everysec

BecauseofRedisGearsrsquoflexibility(itsactuallyrunningpython)youcanachieveinternalmoduleintegrationwithitndash ReadfromhashesandindexinRediSearchRedisGraphndash ReadRedisJSONdataandpasstoRedisTimeSeriesndash hellip

UseCase2ndashaMultiModelEngine

RediSearch

RedisHashesEveryupdate

RedisGraph

createthebuilderbuilder=GearsBuilder()filtereventsonkeyall_keysbuilderfilter(lambdaxx[key]=all_keys)addthekeystoall_keyssetbuildermap(lambdaxexecute(saddall_keysx[key]))registertheexecutiononkeyspacenotificationbuilderregister()

Buildagearthatcreatesmaintainsasetofallkeyswithinredis

Recipe1ndasheventriggering

createthepipebuilderKeysOnlyReaderisaperformanceimprovementonlypipingthekeysbuilder=GearsBuilder(KeysOnlyReader)getfromeachhashthegenresfieldbuildermap(lambdaxexecute(hgetxgenres))filterthosewhodonothavegenresbuilderfilter(lambdaxxisnotNone)splitgenresbycommabuilderflatmap(lambdaxxsplit())countforeachgenrethenumberoftimesitappearsbuildercountby()starttheexecutionbuilderrun(movie)

Buildagearthatcountshowoftenagenreisusedwithinasetofmovies

Recipe2ndashmapreducing

createthebuilderwithaStreamReaderbuilder=GearsBuilder(StreamReader)extracteachfieldvaluepairfromthemessageandincreasethepipegranularitybuilderflatmap(lambdax[(a[0]a[1])forainxitems()])filteroutthestreamIditselfbuilderfilter(lambdaxx[0]=streamId)makesurethegearsdatalivesinthecorrectshardbuilderrepartition(lambdaxx[0])applyeachfieldvaluepairtoakeybuilderforeach(lambdaxexecute(setx[0]x[1]))registeronnewmessagesonthestreaminputStreambuilderregister(inputStream)

Buildagearthatconsumesastreamandupdateskeysaccordingly

Recipe3ndashstreamprocessing

ExampleTriggerExplained

47

ExampleTriggerExplained

48

ExampleTriggerExplained

49

ExampleTriggerExplained-Flatmap

50

ExampleTriggerExplained-Repartition

51

ExampleTriggerExplained-executeCommand

52

Demo

DemoSetup

54

bull httpsgithubcomRedisGearsAnimalRecognitionDemo

bull redisfoundmycat

Challenge

55

33

RedisGears

Introducing

RedisGearsisaServerlessengineformulti-modelandclusteroperationsinRedissupportingbotheventdriven

aswellasbatchoperations

WhatisRedisGears

GearsCoordinator MapReducerGearsExecuter

GearsinfrastructureiswritteninC

C-API

Soon Soon

HighPerformanceArchitecture

RedisGearsallowstodefineapipeofoperations

bull Returningvaluefromoneoperationispassedtothenextoperationinthepipe

bull Lastoperationreturningtheresulttotheuserbull Firstoperationiscalledlsquoreaderrsquo-responsibleforprovidingdata

ndash Keysreader-readkeysfromRedisndash Streamreader-readstreamsfromRedisndash Pythonreader-allowtousertowritehisownreadersinpython

ScriptingwithRedisGears

36

bull Map

bull FlatMap

bull Filterbull Groupby+Reducebull Aggregatebull Sortbull Limit

bull ForEachbull Distinct

SupportedOperations

37

Reader(Flat)Mapper

UsingRedisGearsndash(Flat)Mapping

UsingRedisGears-Filtering

Reader Filterrecordwith1doc

UsingRedisGears-Aggregate

Reader CountAggregator

1

1

Demo

GearshasastreamingAPItoallowtotriggergearsexecutiononeventsndash RedisStreamevents-Triggeranexecutionwheneveranewdataentersasteamndash RedisKeysevents-Triggeranoperationwheneverakeyistouched

UseCase1ndashStreamProcessing

RedisTimeSeriesRedisStreams

Everysec

BecauseofRedisGearsrsquoflexibility(itsactuallyrunningpython)youcanachieveinternalmoduleintegrationwithitndash ReadfromhashesandindexinRediSearchRedisGraphndash ReadRedisJSONdataandpasstoRedisTimeSeriesndash hellip

UseCase2ndashaMultiModelEngine

RediSearch

RedisHashesEveryupdate

RedisGraph

createthebuilderbuilder=GearsBuilder()filtereventsonkeyall_keysbuilderfilter(lambdaxx[key]=all_keys)addthekeystoall_keyssetbuildermap(lambdaxexecute(saddall_keysx[key]))registertheexecutiononkeyspacenotificationbuilderregister()

Buildagearthatcreatesmaintainsasetofallkeyswithinredis

Recipe1ndasheventriggering

createthepipebuilderKeysOnlyReaderisaperformanceimprovementonlypipingthekeysbuilder=GearsBuilder(KeysOnlyReader)getfromeachhashthegenresfieldbuildermap(lambdaxexecute(hgetxgenres))filterthosewhodonothavegenresbuilderfilter(lambdaxxisnotNone)splitgenresbycommabuilderflatmap(lambdaxxsplit())countforeachgenrethenumberoftimesitappearsbuildercountby()starttheexecutionbuilderrun(movie)

Buildagearthatcountshowoftenagenreisusedwithinasetofmovies

Recipe2ndashmapreducing

createthebuilderwithaStreamReaderbuilder=GearsBuilder(StreamReader)extracteachfieldvaluepairfromthemessageandincreasethepipegranularitybuilderflatmap(lambdax[(a[0]a[1])forainxitems()])filteroutthestreamIditselfbuilderfilter(lambdaxx[0]=streamId)makesurethegearsdatalivesinthecorrectshardbuilderrepartition(lambdaxx[0])applyeachfieldvaluepairtoakeybuilderforeach(lambdaxexecute(setx[0]x[1]))registeronnewmessagesonthestreaminputStreambuilderregister(inputStream)

Buildagearthatconsumesastreamandupdateskeysaccordingly

Recipe3ndashstreamprocessing

ExampleTriggerExplained

47

ExampleTriggerExplained

48

ExampleTriggerExplained

49

ExampleTriggerExplained-Flatmap

50

ExampleTriggerExplained-Repartition

51

ExampleTriggerExplained-executeCommand

52

Demo

DemoSetup

54

bull httpsgithubcomRedisGearsAnimalRecognitionDemo

bull redisfoundmycat

Challenge

55

RedisGearsisaServerlessengineformulti-modelandclusteroperationsinRedissupportingbotheventdriven

aswellasbatchoperations

WhatisRedisGears

GearsCoordinator MapReducerGearsExecuter

GearsinfrastructureiswritteninC

C-API

Soon Soon

HighPerformanceArchitecture

RedisGearsallowstodefineapipeofoperations

bull Returningvaluefromoneoperationispassedtothenextoperationinthepipe

bull Lastoperationreturningtheresulttotheuserbull Firstoperationiscalledlsquoreaderrsquo-responsibleforprovidingdata

ndash Keysreader-readkeysfromRedisndash Streamreader-readstreamsfromRedisndash Pythonreader-allowtousertowritehisownreadersinpython

ScriptingwithRedisGears

36

bull Map

bull FlatMap

bull Filterbull Groupby+Reducebull Aggregatebull Sortbull Limit

bull ForEachbull Distinct

SupportedOperations

37

Reader(Flat)Mapper

UsingRedisGearsndash(Flat)Mapping

UsingRedisGears-Filtering

Reader Filterrecordwith1doc

UsingRedisGears-Aggregate

Reader CountAggregator

1

1

Demo

GearshasastreamingAPItoallowtotriggergearsexecutiononeventsndash RedisStreamevents-Triggeranexecutionwheneveranewdataentersasteamndash RedisKeysevents-Triggeranoperationwheneverakeyistouched

UseCase1ndashStreamProcessing

RedisTimeSeriesRedisStreams

Everysec

BecauseofRedisGearsrsquoflexibility(itsactuallyrunningpython)youcanachieveinternalmoduleintegrationwithitndash ReadfromhashesandindexinRediSearchRedisGraphndash ReadRedisJSONdataandpasstoRedisTimeSeriesndash hellip

UseCase2ndashaMultiModelEngine

RediSearch

RedisHashesEveryupdate

RedisGraph

createthebuilderbuilder=GearsBuilder()filtereventsonkeyall_keysbuilderfilter(lambdaxx[key]=all_keys)addthekeystoall_keyssetbuildermap(lambdaxexecute(saddall_keysx[key]))registertheexecutiononkeyspacenotificationbuilderregister()

Buildagearthatcreatesmaintainsasetofallkeyswithinredis

Recipe1ndasheventriggering

createthepipebuilderKeysOnlyReaderisaperformanceimprovementonlypipingthekeysbuilder=GearsBuilder(KeysOnlyReader)getfromeachhashthegenresfieldbuildermap(lambdaxexecute(hgetxgenres))filterthosewhodonothavegenresbuilderfilter(lambdaxxisnotNone)splitgenresbycommabuilderflatmap(lambdaxxsplit())countforeachgenrethenumberoftimesitappearsbuildercountby()starttheexecutionbuilderrun(movie)

Buildagearthatcountshowoftenagenreisusedwithinasetofmovies

Recipe2ndashmapreducing

createthebuilderwithaStreamReaderbuilder=GearsBuilder(StreamReader)extracteachfieldvaluepairfromthemessageandincreasethepipegranularitybuilderflatmap(lambdax[(a[0]a[1])forainxitems()])filteroutthestreamIditselfbuilderfilter(lambdaxx[0]=streamId)makesurethegearsdatalivesinthecorrectshardbuilderrepartition(lambdaxx[0])applyeachfieldvaluepairtoakeybuilderforeach(lambdaxexecute(setx[0]x[1]))registeronnewmessagesonthestreaminputStreambuilderregister(inputStream)

Buildagearthatconsumesastreamandupdateskeysaccordingly

Recipe3ndashstreamprocessing

ExampleTriggerExplained

47

ExampleTriggerExplained

48

ExampleTriggerExplained

49

ExampleTriggerExplained-Flatmap

50

ExampleTriggerExplained-Repartition

51

ExampleTriggerExplained-executeCommand

52

Demo

DemoSetup

54

bull httpsgithubcomRedisGearsAnimalRecognitionDemo

bull redisfoundmycat

Challenge

55

GearsCoordinator MapReducerGearsExecuter

GearsinfrastructureiswritteninC

C-API

Soon Soon

HighPerformanceArchitecture

RedisGearsallowstodefineapipeofoperations

bull Returningvaluefromoneoperationispassedtothenextoperationinthepipe

bull Lastoperationreturningtheresulttotheuserbull Firstoperationiscalledlsquoreaderrsquo-responsibleforprovidingdata

ndash Keysreader-readkeysfromRedisndash Streamreader-readstreamsfromRedisndash Pythonreader-allowtousertowritehisownreadersinpython

ScriptingwithRedisGears

36

bull Map

bull FlatMap

bull Filterbull Groupby+Reducebull Aggregatebull Sortbull Limit

bull ForEachbull Distinct

SupportedOperations

37

Reader(Flat)Mapper

UsingRedisGearsndash(Flat)Mapping

UsingRedisGears-Filtering

Reader Filterrecordwith1doc

UsingRedisGears-Aggregate

Reader CountAggregator

1

1

Demo

GearshasastreamingAPItoallowtotriggergearsexecutiononeventsndash RedisStreamevents-Triggeranexecutionwheneveranewdataentersasteamndash RedisKeysevents-Triggeranoperationwheneverakeyistouched

UseCase1ndashStreamProcessing

RedisTimeSeriesRedisStreams

Everysec

BecauseofRedisGearsrsquoflexibility(itsactuallyrunningpython)youcanachieveinternalmoduleintegrationwithitndash ReadfromhashesandindexinRediSearchRedisGraphndash ReadRedisJSONdataandpasstoRedisTimeSeriesndash hellip

UseCase2ndashaMultiModelEngine

RediSearch

RedisHashesEveryupdate

RedisGraph

createthebuilderbuilder=GearsBuilder()filtereventsonkeyall_keysbuilderfilter(lambdaxx[key]=all_keys)addthekeystoall_keyssetbuildermap(lambdaxexecute(saddall_keysx[key]))registertheexecutiononkeyspacenotificationbuilderregister()

Buildagearthatcreatesmaintainsasetofallkeyswithinredis

Recipe1ndasheventriggering

createthepipebuilderKeysOnlyReaderisaperformanceimprovementonlypipingthekeysbuilder=GearsBuilder(KeysOnlyReader)getfromeachhashthegenresfieldbuildermap(lambdaxexecute(hgetxgenres))filterthosewhodonothavegenresbuilderfilter(lambdaxxisnotNone)splitgenresbycommabuilderflatmap(lambdaxxsplit())countforeachgenrethenumberoftimesitappearsbuildercountby()starttheexecutionbuilderrun(movie)

Buildagearthatcountshowoftenagenreisusedwithinasetofmovies

Recipe2ndashmapreducing

createthebuilderwithaStreamReaderbuilder=GearsBuilder(StreamReader)extracteachfieldvaluepairfromthemessageandincreasethepipegranularitybuilderflatmap(lambdax[(a[0]a[1])forainxitems()])filteroutthestreamIditselfbuilderfilter(lambdaxx[0]=streamId)makesurethegearsdatalivesinthecorrectshardbuilderrepartition(lambdaxx[0])applyeachfieldvaluepairtoakeybuilderforeach(lambdaxexecute(setx[0]x[1]))registeronnewmessagesonthestreaminputStreambuilderregister(inputStream)

Buildagearthatconsumesastreamandupdateskeysaccordingly

Recipe3ndashstreamprocessing

ExampleTriggerExplained

47

ExampleTriggerExplained

48

ExampleTriggerExplained

49

ExampleTriggerExplained-Flatmap

50

ExampleTriggerExplained-Repartition

51

ExampleTriggerExplained-executeCommand

52

Demo

DemoSetup

54

bull httpsgithubcomRedisGearsAnimalRecognitionDemo

bull redisfoundmycat

Challenge

55

RedisGearsallowstodefineapipeofoperations

bull Returningvaluefromoneoperationispassedtothenextoperationinthepipe

bull Lastoperationreturningtheresulttotheuserbull Firstoperationiscalledlsquoreaderrsquo-responsibleforprovidingdata

ndash Keysreader-readkeysfromRedisndash Streamreader-readstreamsfromRedisndash Pythonreader-allowtousertowritehisownreadersinpython

ScriptingwithRedisGears

36

bull Map

bull FlatMap

bull Filterbull Groupby+Reducebull Aggregatebull Sortbull Limit

bull ForEachbull Distinct

SupportedOperations

37

Reader(Flat)Mapper

UsingRedisGearsndash(Flat)Mapping

UsingRedisGears-Filtering

Reader Filterrecordwith1doc

UsingRedisGears-Aggregate

Reader CountAggregator

1

1

Demo

GearshasastreamingAPItoallowtotriggergearsexecutiononeventsndash RedisStreamevents-Triggeranexecutionwheneveranewdataentersasteamndash RedisKeysevents-Triggeranoperationwheneverakeyistouched

UseCase1ndashStreamProcessing

RedisTimeSeriesRedisStreams

Everysec

BecauseofRedisGearsrsquoflexibility(itsactuallyrunningpython)youcanachieveinternalmoduleintegrationwithitndash ReadfromhashesandindexinRediSearchRedisGraphndash ReadRedisJSONdataandpasstoRedisTimeSeriesndash hellip

UseCase2ndashaMultiModelEngine

RediSearch

RedisHashesEveryupdate

RedisGraph

createthebuilderbuilder=GearsBuilder()filtereventsonkeyall_keysbuilderfilter(lambdaxx[key]=all_keys)addthekeystoall_keyssetbuildermap(lambdaxexecute(saddall_keysx[key]))registertheexecutiononkeyspacenotificationbuilderregister()

Buildagearthatcreatesmaintainsasetofallkeyswithinredis

Recipe1ndasheventriggering

createthepipebuilderKeysOnlyReaderisaperformanceimprovementonlypipingthekeysbuilder=GearsBuilder(KeysOnlyReader)getfromeachhashthegenresfieldbuildermap(lambdaxexecute(hgetxgenres))filterthosewhodonothavegenresbuilderfilter(lambdaxxisnotNone)splitgenresbycommabuilderflatmap(lambdaxxsplit())countforeachgenrethenumberoftimesitappearsbuildercountby()starttheexecutionbuilderrun(movie)

Buildagearthatcountshowoftenagenreisusedwithinasetofmovies

Recipe2ndashmapreducing

createthebuilderwithaStreamReaderbuilder=GearsBuilder(StreamReader)extracteachfieldvaluepairfromthemessageandincreasethepipegranularitybuilderflatmap(lambdax[(a[0]a[1])forainxitems()])filteroutthestreamIditselfbuilderfilter(lambdaxx[0]=streamId)makesurethegearsdatalivesinthecorrectshardbuilderrepartition(lambdaxx[0])applyeachfieldvaluepairtoakeybuilderforeach(lambdaxexecute(setx[0]x[1]))registeronnewmessagesonthestreaminputStreambuilderregister(inputStream)

Buildagearthatconsumesastreamandupdateskeysaccordingly

Recipe3ndashstreamprocessing

ExampleTriggerExplained

47

ExampleTriggerExplained

48

ExampleTriggerExplained

49

ExampleTriggerExplained-Flatmap

50

ExampleTriggerExplained-Repartition

51

ExampleTriggerExplained-executeCommand

52

Demo

DemoSetup

54

bull httpsgithubcomRedisGearsAnimalRecognitionDemo

bull redisfoundmycat

Challenge

55

bull Map

bull FlatMap

bull Filterbull Groupby+Reducebull Aggregatebull Sortbull Limit

bull ForEachbull Distinct

SupportedOperations

37

Reader(Flat)Mapper

UsingRedisGearsndash(Flat)Mapping

UsingRedisGears-Filtering

Reader Filterrecordwith1doc

UsingRedisGears-Aggregate

Reader CountAggregator

1

1

Demo

GearshasastreamingAPItoallowtotriggergearsexecutiononeventsndash RedisStreamevents-Triggeranexecutionwheneveranewdataentersasteamndash RedisKeysevents-Triggeranoperationwheneverakeyistouched

UseCase1ndashStreamProcessing

RedisTimeSeriesRedisStreams

Everysec

BecauseofRedisGearsrsquoflexibility(itsactuallyrunningpython)youcanachieveinternalmoduleintegrationwithitndash ReadfromhashesandindexinRediSearchRedisGraphndash ReadRedisJSONdataandpasstoRedisTimeSeriesndash hellip

UseCase2ndashaMultiModelEngine

RediSearch

RedisHashesEveryupdate

RedisGraph

createthebuilderbuilder=GearsBuilder()filtereventsonkeyall_keysbuilderfilter(lambdaxx[key]=all_keys)addthekeystoall_keyssetbuildermap(lambdaxexecute(saddall_keysx[key]))registertheexecutiononkeyspacenotificationbuilderregister()

Buildagearthatcreatesmaintainsasetofallkeyswithinredis

Recipe1ndasheventriggering

createthepipebuilderKeysOnlyReaderisaperformanceimprovementonlypipingthekeysbuilder=GearsBuilder(KeysOnlyReader)getfromeachhashthegenresfieldbuildermap(lambdaxexecute(hgetxgenres))filterthosewhodonothavegenresbuilderfilter(lambdaxxisnotNone)splitgenresbycommabuilderflatmap(lambdaxxsplit())countforeachgenrethenumberoftimesitappearsbuildercountby()starttheexecutionbuilderrun(movie)

Buildagearthatcountshowoftenagenreisusedwithinasetofmovies

Recipe2ndashmapreducing

createthebuilderwithaStreamReaderbuilder=GearsBuilder(StreamReader)extracteachfieldvaluepairfromthemessageandincreasethepipegranularitybuilderflatmap(lambdax[(a[0]a[1])forainxitems()])filteroutthestreamIditselfbuilderfilter(lambdaxx[0]=streamId)makesurethegearsdatalivesinthecorrectshardbuilderrepartition(lambdaxx[0])applyeachfieldvaluepairtoakeybuilderforeach(lambdaxexecute(setx[0]x[1]))registeronnewmessagesonthestreaminputStreambuilderregister(inputStream)

Buildagearthatconsumesastreamandupdateskeysaccordingly

Recipe3ndashstreamprocessing

ExampleTriggerExplained

47

ExampleTriggerExplained

48

ExampleTriggerExplained

49

ExampleTriggerExplained-Flatmap

50

ExampleTriggerExplained-Repartition

51

ExampleTriggerExplained-executeCommand

52

Demo

DemoSetup

54

bull httpsgithubcomRedisGearsAnimalRecognitionDemo

bull redisfoundmycat

Challenge

55

Reader(Flat)Mapper

UsingRedisGearsndash(Flat)Mapping

UsingRedisGears-Filtering

Reader Filterrecordwith1doc

UsingRedisGears-Aggregate

Reader CountAggregator

1

1

Demo

GearshasastreamingAPItoallowtotriggergearsexecutiononeventsndash RedisStreamevents-Triggeranexecutionwheneveranewdataentersasteamndash RedisKeysevents-Triggeranoperationwheneverakeyistouched

UseCase1ndashStreamProcessing

RedisTimeSeriesRedisStreams

Everysec

BecauseofRedisGearsrsquoflexibility(itsactuallyrunningpython)youcanachieveinternalmoduleintegrationwithitndash ReadfromhashesandindexinRediSearchRedisGraphndash ReadRedisJSONdataandpasstoRedisTimeSeriesndash hellip

UseCase2ndashaMultiModelEngine

RediSearch

RedisHashesEveryupdate

RedisGraph

createthebuilderbuilder=GearsBuilder()filtereventsonkeyall_keysbuilderfilter(lambdaxx[key]=all_keys)addthekeystoall_keyssetbuildermap(lambdaxexecute(saddall_keysx[key]))registertheexecutiononkeyspacenotificationbuilderregister()

Buildagearthatcreatesmaintainsasetofallkeyswithinredis

Recipe1ndasheventriggering

createthepipebuilderKeysOnlyReaderisaperformanceimprovementonlypipingthekeysbuilder=GearsBuilder(KeysOnlyReader)getfromeachhashthegenresfieldbuildermap(lambdaxexecute(hgetxgenres))filterthosewhodonothavegenresbuilderfilter(lambdaxxisnotNone)splitgenresbycommabuilderflatmap(lambdaxxsplit())countforeachgenrethenumberoftimesitappearsbuildercountby()starttheexecutionbuilderrun(movie)

Buildagearthatcountshowoftenagenreisusedwithinasetofmovies

Recipe2ndashmapreducing

createthebuilderwithaStreamReaderbuilder=GearsBuilder(StreamReader)extracteachfieldvaluepairfromthemessageandincreasethepipegranularitybuilderflatmap(lambdax[(a[0]a[1])forainxitems()])filteroutthestreamIditselfbuilderfilter(lambdaxx[0]=streamId)makesurethegearsdatalivesinthecorrectshardbuilderrepartition(lambdaxx[0])applyeachfieldvaluepairtoakeybuilderforeach(lambdaxexecute(setx[0]x[1]))registeronnewmessagesonthestreaminputStreambuilderregister(inputStream)

Buildagearthatconsumesastreamandupdateskeysaccordingly

Recipe3ndashstreamprocessing

ExampleTriggerExplained

47

ExampleTriggerExplained

48

ExampleTriggerExplained

49

ExampleTriggerExplained-Flatmap

50

ExampleTriggerExplained-Repartition

51

ExampleTriggerExplained-executeCommand

52

Demo

DemoSetup

54

bull httpsgithubcomRedisGearsAnimalRecognitionDemo

bull redisfoundmycat

Challenge

55

UsingRedisGears-Filtering

Reader Filterrecordwith1doc

UsingRedisGears-Aggregate

Reader CountAggregator

1

1

Demo

GearshasastreamingAPItoallowtotriggergearsexecutiononeventsndash RedisStreamevents-Triggeranexecutionwheneveranewdataentersasteamndash RedisKeysevents-Triggeranoperationwheneverakeyistouched

UseCase1ndashStreamProcessing

RedisTimeSeriesRedisStreams

Everysec

BecauseofRedisGearsrsquoflexibility(itsactuallyrunningpython)youcanachieveinternalmoduleintegrationwithitndash ReadfromhashesandindexinRediSearchRedisGraphndash ReadRedisJSONdataandpasstoRedisTimeSeriesndash hellip

UseCase2ndashaMultiModelEngine

RediSearch

RedisHashesEveryupdate

RedisGraph

createthebuilderbuilder=GearsBuilder()filtereventsonkeyall_keysbuilderfilter(lambdaxx[key]=all_keys)addthekeystoall_keyssetbuildermap(lambdaxexecute(saddall_keysx[key]))registertheexecutiononkeyspacenotificationbuilderregister()

Buildagearthatcreatesmaintainsasetofallkeyswithinredis

Recipe1ndasheventriggering

createthepipebuilderKeysOnlyReaderisaperformanceimprovementonlypipingthekeysbuilder=GearsBuilder(KeysOnlyReader)getfromeachhashthegenresfieldbuildermap(lambdaxexecute(hgetxgenres))filterthosewhodonothavegenresbuilderfilter(lambdaxxisnotNone)splitgenresbycommabuilderflatmap(lambdaxxsplit())countforeachgenrethenumberoftimesitappearsbuildercountby()starttheexecutionbuilderrun(movie)

Buildagearthatcountshowoftenagenreisusedwithinasetofmovies

Recipe2ndashmapreducing

createthebuilderwithaStreamReaderbuilder=GearsBuilder(StreamReader)extracteachfieldvaluepairfromthemessageandincreasethepipegranularitybuilderflatmap(lambdax[(a[0]a[1])forainxitems()])filteroutthestreamIditselfbuilderfilter(lambdaxx[0]=streamId)makesurethegearsdatalivesinthecorrectshardbuilderrepartition(lambdaxx[0])applyeachfieldvaluepairtoakeybuilderforeach(lambdaxexecute(setx[0]x[1]))registeronnewmessagesonthestreaminputStreambuilderregister(inputStream)

Buildagearthatconsumesastreamandupdateskeysaccordingly

Recipe3ndashstreamprocessing

ExampleTriggerExplained

47

ExampleTriggerExplained

48

ExampleTriggerExplained

49

ExampleTriggerExplained-Flatmap

50

ExampleTriggerExplained-Repartition

51

ExampleTriggerExplained-executeCommand

52

Demo

DemoSetup

54

bull httpsgithubcomRedisGearsAnimalRecognitionDemo

bull redisfoundmycat

Challenge

55

UsingRedisGears-Aggregate

Reader CountAggregator

1

1

Demo

GearshasastreamingAPItoallowtotriggergearsexecutiononeventsndash RedisStreamevents-Triggeranexecutionwheneveranewdataentersasteamndash RedisKeysevents-Triggeranoperationwheneverakeyistouched

UseCase1ndashStreamProcessing

RedisTimeSeriesRedisStreams

Everysec

BecauseofRedisGearsrsquoflexibility(itsactuallyrunningpython)youcanachieveinternalmoduleintegrationwithitndash ReadfromhashesandindexinRediSearchRedisGraphndash ReadRedisJSONdataandpasstoRedisTimeSeriesndash hellip

UseCase2ndashaMultiModelEngine

RediSearch

RedisHashesEveryupdate

RedisGraph

createthebuilderbuilder=GearsBuilder()filtereventsonkeyall_keysbuilderfilter(lambdaxx[key]=all_keys)addthekeystoall_keyssetbuildermap(lambdaxexecute(saddall_keysx[key]))registertheexecutiononkeyspacenotificationbuilderregister()

Buildagearthatcreatesmaintainsasetofallkeyswithinredis

Recipe1ndasheventriggering

createthepipebuilderKeysOnlyReaderisaperformanceimprovementonlypipingthekeysbuilder=GearsBuilder(KeysOnlyReader)getfromeachhashthegenresfieldbuildermap(lambdaxexecute(hgetxgenres))filterthosewhodonothavegenresbuilderfilter(lambdaxxisnotNone)splitgenresbycommabuilderflatmap(lambdaxxsplit())countforeachgenrethenumberoftimesitappearsbuildercountby()starttheexecutionbuilderrun(movie)

Buildagearthatcountshowoftenagenreisusedwithinasetofmovies

Recipe2ndashmapreducing

createthebuilderwithaStreamReaderbuilder=GearsBuilder(StreamReader)extracteachfieldvaluepairfromthemessageandincreasethepipegranularitybuilderflatmap(lambdax[(a[0]a[1])forainxitems()])filteroutthestreamIditselfbuilderfilter(lambdaxx[0]=streamId)makesurethegearsdatalivesinthecorrectshardbuilderrepartition(lambdaxx[0])applyeachfieldvaluepairtoakeybuilderforeach(lambdaxexecute(setx[0]x[1]))registeronnewmessagesonthestreaminputStreambuilderregister(inputStream)

Buildagearthatconsumesastreamandupdateskeysaccordingly

Recipe3ndashstreamprocessing

ExampleTriggerExplained

47

ExampleTriggerExplained

48

ExampleTriggerExplained

49

ExampleTriggerExplained-Flatmap

50

ExampleTriggerExplained-Repartition

51

ExampleTriggerExplained-executeCommand

52

Demo

DemoSetup

54

bull httpsgithubcomRedisGearsAnimalRecognitionDemo

bull redisfoundmycat

Challenge

55

Demo

GearshasastreamingAPItoallowtotriggergearsexecutiononeventsndash RedisStreamevents-Triggeranexecutionwheneveranewdataentersasteamndash RedisKeysevents-Triggeranoperationwheneverakeyistouched

UseCase1ndashStreamProcessing

RedisTimeSeriesRedisStreams

Everysec

BecauseofRedisGearsrsquoflexibility(itsactuallyrunningpython)youcanachieveinternalmoduleintegrationwithitndash ReadfromhashesandindexinRediSearchRedisGraphndash ReadRedisJSONdataandpasstoRedisTimeSeriesndash hellip

UseCase2ndashaMultiModelEngine

RediSearch

RedisHashesEveryupdate

RedisGraph

createthebuilderbuilder=GearsBuilder()filtereventsonkeyall_keysbuilderfilter(lambdaxx[key]=all_keys)addthekeystoall_keyssetbuildermap(lambdaxexecute(saddall_keysx[key]))registertheexecutiononkeyspacenotificationbuilderregister()

Buildagearthatcreatesmaintainsasetofallkeyswithinredis

Recipe1ndasheventriggering

createthepipebuilderKeysOnlyReaderisaperformanceimprovementonlypipingthekeysbuilder=GearsBuilder(KeysOnlyReader)getfromeachhashthegenresfieldbuildermap(lambdaxexecute(hgetxgenres))filterthosewhodonothavegenresbuilderfilter(lambdaxxisnotNone)splitgenresbycommabuilderflatmap(lambdaxxsplit())countforeachgenrethenumberoftimesitappearsbuildercountby()starttheexecutionbuilderrun(movie)

Buildagearthatcountshowoftenagenreisusedwithinasetofmovies

Recipe2ndashmapreducing

createthebuilderwithaStreamReaderbuilder=GearsBuilder(StreamReader)extracteachfieldvaluepairfromthemessageandincreasethepipegranularitybuilderflatmap(lambdax[(a[0]a[1])forainxitems()])filteroutthestreamIditselfbuilderfilter(lambdaxx[0]=streamId)makesurethegearsdatalivesinthecorrectshardbuilderrepartition(lambdaxx[0])applyeachfieldvaluepairtoakeybuilderforeach(lambdaxexecute(setx[0]x[1]))registeronnewmessagesonthestreaminputStreambuilderregister(inputStream)

Buildagearthatconsumesastreamandupdateskeysaccordingly

Recipe3ndashstreamprocessing

ExampleTriggerExplained

47

ExampleTriggerExplained

48

ExampleTriggerExplained

49

ExampleTriggerExplained-Flatmap

50

ExampleTriggerExplained-Repartition

51

ExampleTriggerExplained-executeCommand

52

Demo

DemoSetup

54

bull httpsgithubcomRedisGearsAnimalRecognitionDemo

bull redisfoundmycat

Challenge

55

GearshasastreamingAPItoallowtotriggergearsexecutiononeventsndash RedisStreamevents-Triggeranexecutionwheneveranewdataentersasteamndash RedisKeysevents-Triggeranoperationwheneverakeyistouched

UseCase1ndashStreamProcessing

RedisTimeSeriesRedisStreams

Everysec

BecauseofRedisGearsrsquoflexibility(itsactuallyrunningpython)youcanachieveinternalmoduleintegrationwithitndash ReadfromhashesandindexinRediSearchRedisGraphndash ReadRedisJSONdataandpasstoRedisTimeSeriesndash hellip

UseCase2ndashaMultiModelEngine

RediSearch

RedisHashesEveryupdate

RedisGraph

createthebuilderbuilder=GearsBuilder()filtereventsonkeyall_keysbuilderfilter(lambdaxx[key]=all_keys)addthekeystoall_keyssetbuildermap(lambdaxexecute(saddall_keysx[key]))registertheexecutiononkeyspacenotificationbuilderregister()

Buildagearthatcreatesmaintainsasetofallkeyswithinredis

Recipe1ndasheventriggering

createthepipebuilderKeysOnlyReaderisaperformanceimprovementonlypipingthekeysbuilder=GearsBuilder(KeysOnlyReader)getfromeachhashthegenresfieldbuildermap(lambdaxexecute(hgetxgenres))filterthosewhodonothavegenresbuilderfilter(lambdaxxisnotNone)splitgenresbycommabuilderflatmap(lambdaxxsplit())countforeachgenrethenumberoftimesitappearsbuildercountby()starttheexecutionbuilderrun(movie)

Buildagearthatcountshowoftenagenreisusedwithinasetofmovies

Recipe2ndashmapreducing

createthebuilderwithaStreamReaderbuilder=GearsBuilder(StreamReader)extracteachfieldvaluepairfromthemessageandincreasethepipegranularitybuilderflatmap(lambdax[(a[0]a[1])forainxitems()])filteroutthestreamIditselfbuilderfilter(lambdaxx[0]=streamId)makesurethegearsdatalivesinthecorrectshardbuilderrepartition(lambdaxx[0])applyeachfieldvaluepairtoakeybuilderforeach(lambdaxexecute(setx[0]x[1]))registeronnewmessagesonthestreaminputStreambuilderregister(inputStream)

Buildagearthatconsumesastreamandupdateskeysaccordingly

Recipe3ndashstreamprocessing

ExampleTriggerExplained

47

ExampleTriggerExplained

48

ExampleTriggerExplained

49

ExampleTriggerExplained-Flatmap

50

ExampleTriggerExplained-Repartition

51

ExampleTriggerExplained-executeCommand

52

Demo

DemoSetup

54

bull httpsgithubcomRedisGearsAnimalRecognitionDemo

bull redisfoundmycat

Challenge

55

BecauseofRedisGearsrsquoflexibility(itsactuallyrunningpython)youcanachieveinternalmoduleintegrationwithitndash ReadfromhashesandindexinRediSearchRedisGraphndash ReadRedisJSONdataandpasstoRedisTimeSeriesndash hellip

UseCase2ndashaMultiModelEngine

RediSearch

RedisHashesEveryupdate

RedisGraph

createthebuilderbuilder=GearsBuilder()filtereventsonkeyall_keysbuilderfilter(lambdaxx[key]=all_keys)addthekeystoall_keyssetbuildermap(lambdaxexecute(saddall_keysx[key]))registertheexecutiononkeyspacenotificationbuilderregister()

Buildagearthatcreatesmaintainsasetofallkeyswithinredis

Recipe1ndasheventriggering

createthepipebuilderKeysOnlyReaderisaperformanceimprovementonlypipingthekeysbuilder=GearsBuilder(KeysOnlyReader)getfromeachhashthegenresfieldbuildermap(lambdaxexecute(hgetxgenres))filterthosewhodonothavegenresbuilderfilter(lambdaxxisnotNone)splitgenresbycommabuilderflatmap(lambdaxxsplit())countforeachgenrethenumberoftimesitappearsbuildercountby()starttheexecutionbuilderrun(movie)

Buildagearthatcountshowoftenagenreisusedwithinasetofmovies

Recipe2ndashmapreducing

createthebuilderwithaStreamReaderbuilder=GearsBuilder(StreamReader)extracteachfieldvaluepairfromthemessageandincreasethepipegranularitybuilderflatmap(lambdax[(a[0]a[1])forainxitems()])filteroutthestreamIditselfbuilderfilter(lambdaxx[0]=streamId)makesurethegearsdatalivesinthecorrectshardbuilderrepartition(lambdaxx[0])applyeachfieldvaluepairtoakeybuilderforeach(lambdaxexecute(setx[0]x[1]))registeronnewmessagesonthestreaminputStreambuilderregister(inputStream)

Buildagearthatconsumesastreamandupdateskeysaccordingly

Recipe3ndashstreamprocessing

ExampleTriggerExplained

47

ExampleTriggerExplained

48

ExampleTriggerExplained

49

ExampleTriggerExplained-Flatmap

50

ExampleTriggerExplained-Repartition

51

ExampleTriggerExplained-executeCommand

52

Demo

DemoSetup

54

bull httpsgithubcomRedisGearsAnimalRecognitionDemo

bull redisfoundmycat

Challenge

55

createthebuilderbuilder=GearsBuilder()filtereventsonkeyall_keysbuilderfilter(lambdaxx[key]=all_keys)addthekeystoall_keyssetbuildermap(lambdaxexecute(saddall_keysx[key]))registertheexecutiononkeyspacenotificationbuilderregister()

Buildagearthatcreatesmaintainsasetofallkeyswithinredis

Recipe1ndasheventriggering

createthepipebuilderKeysOnlyReaderisaperformanceimprovementonlypipingthekeysbuilder=GearsBuilder(KeysOnlyReader)getfromeachhashthegenresfieldbuildermap(lambdaxexecute(hgetxgenres))filterthosewhodonothavegenresbuilderfilter(lambdaxxisnotNone)splitgenresbycommabuilderflatmap(lambdaxxsplit())countforeachgenrethenumberoftimesitappearsbuildercountby()starttheexecutionbuilderrun(movie)

Buildagearthatcountshowoftenagenreisusedwithinasetofmovies

Recipe2ndashmapreducing

createthebuilderwithaStreamReaderbuilder=GearsBuilder(StreamReader)extracteachfieldvaluepairfromthemessageandincreasethepipegranularitybuilderflatmap(lambdax[(a[0]a[1])forainxitems()])filteroutthestreamIditselfbuilderfilter(lambdaxx[0]=streamId)makesurethegearsdatalivesinthecorrectshardbuilderrepartition(lambdaxx[0])applyeachfieldvaluepairtoakeybuilderforeach(lambdaxexecute(setx[0]x[1]))registeronnewmessagesonthestreaminputStreambuilderregister(inputStream)

Buildagearthatconsumesastreamandupdateskeysaccordingly

Recipe3ndashstreamprocessing

ExampleTriggerExplained

47

ExampleTriggerExplained

48

ExampleTriggerExplained

49

ExampleTriggerExplained-Flatmap

50

ExampleTriggerExplained-Repartition

51

ExampleTriggerExplained-executeCommand

52

Demo

DemoSetup

54

bull httpsgithubcomRedisGearsAnimalRecognitionDemo

bull redisfoundmycat

Challenge

55

createthepipebuilderKeysOnlyReaderisaperformanceimprovementonlypipingthekeysbuilder=GearsBuilder(KeysOnlyReader)getfromeachhashthegenresfieldbuildermap(lambdaxexecute(hgetxgenres))filterthosewhodonothavegenresbuilderfilter(lambdaxxisnotNone)splitgenresbycommabuilderflatmap(lambdaxxsplit())countforeachgenrethenumberoftimesitappearsbuildercountby()starttheexecutionbuilderrun(movie)

Buildagearthatcountshowoftenagenreisusedwithinasetofmovies

Recipe2ndashmapreducing

createthebuilderwithaStreamReaderbuilder=GearsBuilder(StreamReader)extracteachfieldvaluepairfromthemessageandincreasethepipegranularitybuilderflatmap(lambdax[(a[0]a[1])forainxitems()])filteroutthestreamIditselfbuilderfilter(lambdaxx[0]=streamId)makesurethegearsdatalivesinthecorrectshardbuilderrepartition(lambdaxx[0])applyeachfieldvaluepairtoakeybuilderforeach(lambdaxexecute(setx[0]x[1]))registeronnewmessagesonthestreaminputStreambuilderregister(inputStream)

Buildagearthatconsumesastreamandupdateskeysaccordingly

Recipe3ndashstreamprocessing

ExampleTriggerExplained

47

ExampleTriggerExplained

48

ExampleTriggerExplained

49

ExampleTriggerExplained-Flatmap

50

ExampleTriggerExplained-Repartition

51

ExampleTriggerExplained-executeCommand

52

Demo

DemoSetup

54

bull httpsgithubcomRedisGearsAnimalRecognitionDemo

bull redisfoundmycat

Challenge

55

createthebuilderwithaStreamReaderbuilder=GearsBuilder(StreamReader)extracteachfieldvaluepairfromthemessageandincreasethepipegranularitybuilderflatmap(lambdax[(a[0]a[1])forainxitems()])filteroutthestreamIditselfbuilderfilter(lambdaxx[0]=streamId)makesurethegearsdatalivesinthecorrectshardbuilderrepartition(lambdaxx[0])applyeachfieldvaluepairtoakeybuilderforeach(lambdaxexecute(setx[0]x[1]))registeronnewmessagesonthestreaminputStreambuilderregister(inputStream)

Buildagearthatconsumesastreamandupdateskeysaccordingly

Recipe3ndashstreamprocessing

ExampleTriggerExplained

47

ExampleTriggerExplained

48

ExampleTriggerExplained

49

ExampleTriggerExplained-Flatmap

50

ExampleTriggerExplained-Repartition

51

ExampleTriggerExplained-executeCommand

52

Demo

DemoSetup

54

bull httpsgithubcomRedisGearsAnimalRecognitionDemo

bull redisfoundmycat

Challenge

55

ExampleTriggerExplained

47

ExampleTriggerExplained

48

ExampleTriggerExplained

49

ExampleTriggerExplained-Flatmap

50

ExampleTriggerExplained-Repartition

51

ExampleTriggerExplained-executeCommand

52

Demo

DemoSetup

54

bull httpsgithubcomRedisGearsAnimalRecognitionDemo

bull redisfoundmycat

Challenge

55

ExampleTriggerExplained

48

ExampleTriggerExplained

49

ExampleTriggerExplained-Flatmap

50

ExampleTriggerExplained-Repartition

51

ExampleTriggerExplained-executeCommand

52

Demo

DemoSetup

54

bull httpsgithubcomRedisGearsAnimalRecognitionDemo

bull redisfoundmycat

Challenge

55

ExampleTriggerExplained

49

ExampleTriggerExplained-Flatmap

50

ExampleTriggerExplained-Repartition

51

ExampleTriggerExplained-executeCommand

52

Demo

DemoSetup

54

bull httpsgithubcomRedisGearsAnimalRecognitionDemo

bull redisfoundmycat

Challenge

55

ExampleTriggerExplained-Flatmap

50

ExampleTriggerExplained-Repartition

51

ExampleTriggerExplained-executeCommand

52

Demo

DemoSetup

54

bull httpsgithubcomRedisGearsAnimalRecognitionDemo

bull redisfoundmycat

Challenge

55

ExampleTriggerExplained-Repartition

51

ExampleTriggerExplained-executeCommand

52

Demo

DemoSetup

54

bull httpsgithubcomRedisGearsAnimalRecognitionDemo

bull redisfoundmycat

Challenge

55

ExampleTriggerExplained-executeCommand

52

Demo

DemoSetup

54

bull httpsgithubcomRedisGearsAnimalRecognitionDemo

bull redisfoundmycat

Challenge

55

Demo

DemoSetup

54

bull httpsgithubcomRedisGearsAnimalRecognitionDemo

bull redisfoundmycat

Challenge

55

DemoSetup

54

bull httpsgithubcomRedisGearsAnimalRecognitionDemo

bull redisfoundmycat

Challenge

55

bull httpsgithubcomRedisGearsAnimalRecognitionDemo

bull redisfoundmycat

Challenge

55

RediSearch(GA) redisearchio

RedisBloom(GA) redisbloomio

RedisTimeSeries redistimeseriesio

RedisJSON(GA) redisjsonio

RedisAI redisaiio

RedisGraph(GA) redisgraphio

RedisModules

56

RedisGearsredisgearsio

Thankyoupieterredislabscom

57