Modularized Persistence - B Zsoldos
-
Upload
mfrancis -
Category
Technology
-
view
158 -
download
1
description
Transcript of Modularized Persistence - B Zsoldos
![Page 1: Modularized Persistence - B Zsoldos](https://reader034.fdocuments.us/reader034/viewer/2022052508/5597968a1a28ab73538b45d6/html5/thumbnails/1.jpg)
Balázs Zsoldos
Modularized Persistence
![Page 2: Modularized Persistence - B Zsoldos](https://reader034.fdocuments.us/reader034/viewer/2022052508/5597968a1a28ab73538b45d6/html5/thumbnails/2.jpg)
History of NoSQL by Mark Madsen, Picture published by Edd Dumbill
![Page 3: Modularized Persistence - B Zsoldos](https://reader034.fdocuments.us/reader034/viewer/2022052508/5597968a1a28ab73538b45d6/html5/thumbnails/3.jpg)
Blueprint+
JPA+
JSF
DeclarativeServices
+ModularizedPersistence
+JSF
ECM+
ModularizedPersistence
+JSF
ECM+
ModularizedPersistence
+Modularized
Web
DeclarativeServices
+JPA
+JSF
![Page 4: Modularized Persistence - B Zsoldos](https://reader034.fdocuments.us/reader034/viewer/2022052508/5597968a1a28ab73538b45d6/html5/thumbnails/4.jpg)
Stefan Seifert – Apache Sling & Friends Tech Meetup, Berlin 2012
![Page 5: Modularized Persistence - B Zsoldos](https://reader034.fdocuments.us/reader034/viewer/2022052508/5597968a1a28ab73538b45d6/html5/thumbnails/5.jpg)
JDBC Driver(DataSourceFactory)
![Page 6: Modularized Persistence - B Zsoldos](https://reader034.fdocuments.us/reader034/viewer/2022052508/5597968a1a28ab73538b45d6/html5/thumbnails/6.jpg)
JDBC Driver(DataSourceFactory)
DSF Component(XADataSource)
![Page 7: Modularized Persistence - B Zsoldos](https://reader034.fdocuments.us/reader034/viewer/2022052508/5597968a1a28ab73538b45d6/html5/thumbnails/7.jpg)
![Page 8: Modularized Persistence - B Zsoldos](https://reader034.fdocuments.us/reader034/viewer/2022052508/5597968a1a28ab73538b45d6/html5/thumbnails/8.jpg)
JDBC Driver(DataSourceFactory)
DSF Component(XADataSource)
DBCP Component(DataSource)
![Page 9: Modularized Persistence - B Zsoldos](https://reader034.fdocuments.us/reader034/viewer/2022052508/5597968a1a28ab73538b45d6/html5/thumbnails/9.jpg)
![Page 10: Modularized Persistence - B Zsoldos](https://reader034.fdocuments.us/reader034/viewer/2022052508/5597968a1a28ab73538b45d6/html5/thumbnails/10.jpg)
JDBC Driver(DataSourceFactory)
DSF Component(XADataSource)
DBCP Component(DataSource)
???
![Page 11: Modularized Persistence - B Zsoldos](https://reader034.fdocuments.us/reader034/viewer/2022052508/5597968a1a28ab73538b45d6/html5/thumbnails/11.jpg)
JDBC Driver(DataSourceFactory)
DSF Component(XADataSource)
DBCP Component(DataSource)
Liquibase DataSource Component(DataSource)
![Page 12: Modularized Persistence - B Zsoldos](https://reader034.fdocuments.us/reader034/viewer/2022052508/5597968a1a28ab73538b45d6/html5/thumbnails/12.jpg)
![Page 13: Modularized Persistence - B Zsoldos](https://reader034.fdocuments.us/reader034/viewer/2022052508/5597968a1a28ab73538b45d6/html5/thumbnails/13.jpg)
<databaseChangeLog objectQuotingStrategy="QUOTE_ALL_OBJECTS" logicalFilePath="org.everit.osgi.resource.ri.schema" xmlns="http://www.liquibase.org/xml/ns/dbchangelog" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.liquibase.org/xml/ns/dbchangelog http://www.liquibase.org/xml/ns/dbchangelog/dbchangelog-3.0.xsd">
<changeSet id="1.0.0" author="everit">
<createTable tableName="res_resource"> <column name="resource_id" type="bigint" autoIncrement="true"> <constraints primaryKeyName="pk_res_resource" primaryKey="true" nullable="false" /> </column> </createTable> </changeSet>
</databaseChangeLog>
![Page 14: Modularized Persistence - B Zsoldos](https://reader034.fdocuments.us/reader034/viewer/2022052508/5597968a1a28ab73538b45d6/html5/thumbnails/14.jpg)
Provide-Capability: liquibase.schema; name="org.everit.osgi.resource.ri"; resource="/META-INF/liquibase/resource.ri.liquibase.xml";
Provide-Capability: liquibase.schema;name="org.everit.osgi.resource.ri"; resource="/META-INF/liquibase/resource.ri.liquibase.xml"
![Page 15: Modularized Persistence - B Zsoldos](https://reader034.fdocuments.us/reader034/viewer/2022052508/5597968a1a28ab73538b45d6/html5/thumbnails/15.jpg)
JDBC Driver(DataSourceFactory)
DSF Component(XADataSource)
DBCP Component(DataSource)
Liquibase DataSource Component(DataSource)
Business Component Business Component Business ComponentQuerydsl Querydsl Querydsl
![Page 16: Modularized Persistence - B Zsoldos](https://reader034.fdocuments.us/reader034/viewer/2022052508/5597968a1a28ab73538b45d6/html5/thumbnails/16.jpg)
Examples from http://querydsl.com
![Page 17: Modularized Persistence - B Zsoldos](https://reader034.fdocuments.us/reader034/viewer/2022052508/5597968a1a28ab73538b45d6/html5/thumbnails/17.jpg)
Write Liquibase changelog
Write LQMG file
Add LQMG to the Capability
Add inclusions
Generate Querydsl Metadata
![Page 18: Modularized Persistence - B Zsoldos](https://reader034.fdocuments.us/reader034/viewer/2022052508/5597968a1a28ab73538b45d6/html5/thumbnails/18.jpg)
<lqmg xmlns="http://everit.org/lqmg" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" defaultPackage="org.everit.osgi.resource.ri.schema.qdsl"> <namingRules> <classNameRule> <entity>res_resource</entity> <class>Resource</class> <propertyMappings> <primaryKey> <name>pk_res_resource</name> <property>resourcePk</property> </primaryKey> </propertyMappings> </classNameRule> </namingRules></lqmg>
![Page 19: Modularized Persistence - B Zsoldos](https://reader034.fdocuments.us/reader034/viewer/2022052508/5597968a1a28ab73538b45d6/html5/thumbnails/19.jpg)
Write Liquibase changelog
Write LQMG file
Add LQMG to the Capability
Add inclusions
Generate Querydsl Metadata
![Page 20: Modularized Persistence - B Zsoldos](https://reader034.fdocuments.us/reader034/viewer/2022052508/5597968a1a28ab73538b45d6/html5/thumbnails/20.jpg)
Provide-Capability: liquibase.schema; name="org.everit.osgi.resource.ri"; resource="/META-INF/liquibase/resource.ri.liquibase.xml";
Provide-Capability: liquibase.schema;name="org.everit.osgi.resource.ri"; resource="/META-INF/liquibase/resource.ri.liquibase.xml"
![Page 21: Modularized Persistence - B Zsoldos](https://reader034.fdocuments.us/reader034/viewer/2022052508/5597968a1a28ab73538b45d6/html5/thumbnails/21.jpg)
Provide-Capability: liquibase.schema; name="org.everit.osgi.resource.ri"; resource="/META-INF/liquibase/resource.ri.liquibase.xml"; lqmg.config.resource="/META-INF/liquibase/resource.ri.lqmg.xml"
Provide-Capability: liquibase.schema;name="org.everit.osgi.resource.ri"; resource="/META-INF/liquibase/resource.ri.liquibase.xml";lqmg.config.re source="/META-INF/liquibase/resource.ri.lqmg.xml"
![Page 22: Modularized Persistence - B Zsoldos](https://reader034.fdocuments.us/reader034/viewer/2022052508/5597968a1a28ab73538b45d6/html5/thumbnails/22.jpg)
Write Liquibase changelog
Write LQMG file
Add LQMG to the Capability
Add inclusions
Generate Querydsl Metadata
![Page 23: Modularized Persistence - B Zsoldos](https://reader034.fdocuments.us/reader034/viewer/2022052508/5597968a1a28ab73538b45d6/html5/thumbnails/23.jpg)
<databaseChangeLog objectQuotingStrategy="QUOTE_ALL_OBJECTS" logicalFilePath="org.everit.osgi.authorization.ri.schema" xmlns="http://www.liquibase.org/xml/ns/dbchangelog" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:ext="http://www.liquibase.org/xml/ns/dbchangelog-ext" xsi:schemaLocation="http://www.liquibase.org/xml/ns/dbchangelog http://www.liquibase.org/xml/ns/dbchangelog/dbchangelog-3.0.xsd http://www.liquibase.org/xml/ns/dbchangelog-ext http://www.liquibase.org/xml/ns/dbchangelog/dbchangelog-ext.xsd">
<include file="eosgi:org.everit.osgi.resource.ri" /> <include file="eosgi:org.everit.osgi.props.ri" />
<changeSet id="1.0.0" author="everit">
<createTable tableName="authr_permission"> <column name="authorized_resource_id" type="bigint"> <constraints nullable="false" foreignKeyName="fk_res_r_authr_p_a" referencedTableName="res_resource" referencedColumnNames="resource_id" /> </column> <column name="target_resource_id" type="bigint"> <constraints nullable="false" foreignKeyName="fk_res_r_authr_p_t" referencedTableName="res_resource" referencedColumnNames="resource_id" /> </column> <column name="action_" type="varchar(255)"> <constraints nullable="false" /> </column> </createTable> <addPrimaryKey constraintName="pk_authr_perm" tableName="authr_permission" columnNames="authorized_resource_id,target_resource_id,action_" />
<createTable tableName="authr_permission_inheritance"> <column name="parent_resource_id" type="bigint"> <constraints nullable="false" foreignKeyName="fk_res_r_authr_pi_p" referencedTableName="res_resource" referencedColumnNames="resource_id" /> </column> <column name="child_resource_id" type="bigint"> <constraints nullable="false" foreignKeyName="fk_res_r_authr_pi_c" referencedTableName="res_resource" referencedColumnNames="resource_id" /> </column> </createTable> <addPrimaryKey constraintName="pk_authr_perm_inheritance" tableName="authr_permission_inheritance" columnNames="parent_resource_id,child_resource_id" /> </changeSet>
</databaseChangeLog>
<include file="eosgi:org.everit.osgi.resource.ri" /> <include file="eosgi:org.everit.osgi.props.ri" />
![Page 24: Modularized Persistence - B Zsoldos](https://reader034.fdocuments.us/reader034/viewer/2022052508/5597968a1a28ab73538b45d6/html5/thumbnails/24.jpg)
Resource
Authorization
Provide-Capability: liquibase.schema; name="org.everit.osgi.resource.ri"; ...
Require-Capability: liquibase.schema; filter:=(name=org.everit.osgi.resource.ri)
<databaseChangeLog ...>
<include file="eosgi:org.everit.osgi.resource.ri" />
...
</databaseChangeLog>
![Page 25: Modularized Persistence - B Zsoldos](https://reader034.fdocuments.us/reader034/viewer/2022052508/5597968a1a28ab73538b45d6/html5/thumbnails/25.jpg)
Write Liquibase changelog
Write LQMG file
Add LQMG to the Capability
Add inclusions
Generate Querydsl Metadata
![Page 26: Modularized Persistence - B Zsoldos](https://reader034.fdocuments.us/reader034/viewer/2022052508/5597968a1a28ab73538b45d6/html5/thumbnails/26.jpg)
Start embeddedOSGi container
Deploy modules
Start embeddedH2 database
Initialize databaseschema
Generate QuerydslMetadata
LQMG
Evil magic here!!!
If Capability is not found after deployments, unsatisfied bundles are enhanced in the way that unsatisfied requirements are marked to be optional.
![Page 27: Modularized Persistence - B Zsoldos](https://reader034.fdocuments.us/reader034/viewer/2022052508/5597968a1a28ab73538b45d6/html5/thumbnails/27.jpg)
Use-cases???
![Page 28: Modularized Persistence - B Zsoldos](https://reader034.fdocuments.us/reader034/viewer/2022052508/5597968a1a28ab73538b45d6/html5/thumbnails/28.jpg)
Monoholitic
Authorization
Authentication
User
Documents
Articles
SiteMap
Blog
Portal(v5.1.2)
Modularized
Portal
Authorization (v1.1.0)
Authentication (v2.0.0)
User (v1.3.2)
Documents (v1.0.1)
SiteMap (v1.0.3)
Articles (v3.0.14)
Blog (v2.11.0)
![Page 29: Modularized Persistence - B Zsoldos](https://reader034.fdocuments.us/reader034/viewer/2022052508/5597968a1a28ab73538b45d6/html5/thumbnails/29.jpg)
User
User Address
User Address Country
User Address Country
Company
![Page 30: Modularized Persistence - B Zsoldos](https://reader034.fdocuments.us/reader034/viewer/2022052508/5597968a1a28ab73538b45d6/html5/thumbnails/30.jpg)
REAL PROJECT!!!
![Page 31: Modularized Persistence - B Zsoldos](https://reader034.fdocuments.us/reader034/viewer/2022052508/5597968a1a28ab73538b45d6/html5/thumbnails/31.jpg)
Authorization
![Page 32: Modularized Persistence - B Zsoldos](https://reader034.fdocuments.us/reader034/viewer/2022052508/5597968a1a28ab73538b45d6/html5/thumbnails/32.jpg)
Permission
● authorized_resource_id● action● target_resource_id
Resource
● resource_id
Permission Inheritance
● parent_resource_id● child_resource_id
![Page 33: Modularized Persistence - B Zsoldos](https://reader034.fdocuments.us/reader034/viewer/2022052508/5597968a1a28ab73538b45d6/html5/thumbnails/33.jpg)
SELECT ... FROM document d JOIN document.attachment a WHERE … LIMIT 10 OFFSET 1000;
EXISTS(SELECT 1 FROM permission p WHERE p.authorized_resource_id IN (?, …, ?) AND p.target_resource_id = d.resource_id AND action = ?)
SELECT ... FROM document d JOIN document.attachment a WHERE EXISTS(SELECT 1 FROM permission p WHERE p.authorized_resource_id IN (?, …, ?) AND p.target_resource_id = d.resource_id AND action = ?) AND … LIMIT 10 OFFSET 1000;
![Page 34: Modularized Persistence - B Zsoldos](https://reader034.fdocuments.us/reader034/viewer/2022052508/5597968a1a28ab73538b45d6/html5/thumbnails/34.jpg)
@Override public BooleanExpression authorizationPredicate(final long authorizedResourceId, final Expression<Long> targetResourceId, final String... actions) { if (authorizedResourceId == systemResourceId) { return BooleanTemplate.TRUE; }
Objects.requireNonNull(targetResourceId, "Parameter targetResourceId must not be null"); validateActionsParameter(actions);
long[] authorizationScope = getAuthorizationScope(authorizedResourceId);
SQLSubQuery subQuery = new SQLSubQuery();
QPermission permission = QPermission.permission;
BooleanExpression authorizedResourceIdPredicate; if (authorizationScope.length == 1) { authorizedResourceIdPredicate = permission.authorizedResourceId.eq(authorizationScope[0]); } else { // More than one as the scope contains at least one value (other branch) Long[] authorizationScopeLongArray = new Long[authorizationScope.length]; for (int i = 0, n = authorizationScope.length; i < n; i++) { if (authorizationScope[i] == systemResourceId) { return BooleanTemplate.TRUE; } authorizationScopeLongArray[i] = authorizationScope[i]; }
authorizedResourceIdPredicate = permission.authorizedResourceId.in(authorizationScopeLongArray); }
BooleanExpression actionPredicate = null;
if (actions.length == 1) { actionPredicate = permission.action.eq(actions[0]); } else { actionPredicate = permission.action.in(actions); }
return subQuery.from(permission) .where(permission.targetResourceId.eq(targetResourceId).and( actionPredicate.and(authorizedResourceIdPredicate))) .exists(); }
![Page 35: Modularized Persistence - B Zsoldos](https://reader034.fdocuments.us/reader034/viewer/2022052508/5597968a1a28ab73538b45d6/html5/thumbnails/35.jpg)
SQLQuery query = new SQLQuery(connection, configuration);QDocument document = new QDocument("d");
BooleanExpression permissionCheckPredicate = authorizationQdslUtil .authorizationPredicate(userId, document.resourceId, actions);
List<Long> list = query.from(document).where(permissionCheckPredicate) .list(document.documentId);
![Page 36: Modularized Persistence - B Zsoldos](https://reader034.fdocuments.us/reader034/viewer/2022052508/5597968a1a28ab73538b45d6/html5/thumbnails/36.jpg)
JDBC Driver(DataSourceFactory)
DSF Component(XADataSource)
DBCP Component(DataSource)
Liquibase DataSource Component(DataSource)
Business Component Business Component Business Component
QuerydslSupport
QuerydslConfiguration
QuerydslSQLTemplates
Querydsl Querydsl Querydsl
![Page 37: Modularized Persistence - B Zsoldos](https://reader034.fdocuments.us/reader034/viewer/2022052508/5597968a1a28ab73538b45d6/html5/thumbnails/37.jpg)
public List<Long> listDocumentIds() {
return qdsl.execute((connection, configuration) -> {
SQLQuery query = new SQLQuery(connection, configuration); QDocument document = new QDocument("d");
BooleanExpression permissionCheckPredicate = authorizationQdslUtil .authorizationPredicate(a1, document.resourceId, actions);
List<Long> list = query.from(document).where(permissionCheckPredicate) .list(document.documentId); }}
@Reference(name = "querydslSupport", bind = "setQdsl")private QuerydslSupport qdsl;
public void setQdsl(QuerydslSupport qdsl) { this.qdsl = qdsl;}
![Page 38: Modularized Persistence - B Zsoldos](https://reader034.fdocuments.us/reader034/viewer/2022052508/5597968a1a28ab73538b45d6/html5/thumbnails/38.jpg)
JDBC Driver(DataSourceFactory)
DSF Component(XADataSource)
DBCP Component(DataSource)
Liquibase DataSource Component(DataSource)
Business Component Business Component Business Component
QuerydslSupport
QuerydslConfiguration
QuerydslSQLTemplates
Querydsl Querydsl Querydsl
Transaction Helper
![Page 39: Modularized Persistence - B Zsoldos](https://reader034.fdocuments.us/reader034/viewer/2022052508/5597968a1a28ab73538b45d6/html5/thumbnails/39.jpg)
public long saveUser(String firstName, String lastName) { Objects.requireNonNull(firstName); Objects.requireNonNull(lastName);
return transactionHelper.required(() -> { // Logic that should be implemented return 0l; }); }
![Page 40: Modularized Persistence - B Zsoldos](https://reader034.fdocuments.us/reader034/viewer/2022052508/5597968a1a28ab73538b45d6/html5/thumbnails/40.jpg)
Caching
● Available Cache modules: cache-api, cache-infinispan, cache-noop
● Caching should be done in the persistent module by the programmer, who knows the business logic
● Caching and table updates should be done within the same component
● In case bulk update is done in another module, cache must be cleared
![Page 41: Modularized Persistence - B Zsoldos](https://reader034.fdocuments.us/reader034/viewer/2022052508/5597968a1a28ab73538b45d6/html5/thumbnails/41.jpg)
OpenSource modules
● Resource● Authorization● Authentication● Property Manager● Blobstore (Before review)● Audit (Before review)
![Page 42: Modularized Persistence - B Zsoldos](https://reader034.fdocuments.us/reader034/viewer/2022052508/5597968a1a28ab73538b45d6/html5/thumbnails/42.jpg)
Roadmap
![Page 43: Modularized Persistence - B Zsoldos](https://reader034.fdocuments.us/reader034/viewer/2022052508/5597968a1a28ab73538b45d6/html5/thumbnails/43.jpg)
HTML Output
LQMG / Maven
H2
Liquibase schema(Bundle-Capability)
HTML
![Page 44: Modularized Persistence - B Zsoldos](https://reader034.fdocuments.us/reader034/viewer/2022052508/5597968a1a28ab73538b45d6/html5/thumbnails/44.jpg)
![Page 45: Modularized Persistence - B Zsoldos](https://reader034.fdocuments.us/reader034/viewer/2022052508/5597968a1a28ab73538b45d6/html5/thumbnails/45.jpg)
LQMG / Maven
Database
Liquibase schema(Bundle-Capability)
Generate Schema
Manual Schema update
![Page 46: Modularized Persistence - B Zsoldos](https://reader034.fdocuments.us/reader034/viewer/2022052508/5597968a1a28ab73538b45d6/html5/thumbnails/46.jpg)
Webconsole plugin
DataSource(OSGi service)
Liquibase schema(Bundle-Capability)
Generate SchemaCustom User / Password
Manual Schema generation
![Page 47: Modularized Persistence - B Zsoldos](https://reader034.fdocuments.us/reader034/viewer/2022052508/5597968a1a28ab73538b45d6/html5/thumbnails/47.jpg)
Dump SQL file
Webconsole plugin
DataSource(OSGi service)
Liquibase schema(Bundle-Capability)
SQL
![Page 48: Modularized Persistence - B Zsoldos](https://reader034.fdocuments.us/reader034/viewer/2022052508/5597968a1a28ab73538b45d6/html5/thumbnails/48.jpg)
LQMG extension in changelog.xml
<databaseChangeLog objectQuotingStrategy="QUOTE_ALL_OBJECTS" logicalFilePath="org.everit.osgi.resource.ri.schema" xmlns="http://www.liquibase.org/xml/ns/dbchangelog" xmlns:lqmg="http://everit.org/xml/ns/lqmg" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.liquibase.org/xml/ns/dbchangelog http://www.liquibase.org/xml/ns/dbchangelog/dbchangelog-3.0.xsd">
<changeSet id="1.0.0" author="everit">
<createTable tableName="res_resource" lqmg:class="QResource"> <column name="resource_id" type="bigint" autoIncrement="true" lqmg:property="resourceId"> <constraints primaryKeyName="pk_res_resource" primaryKey="true" nullable="false" /> </column> </createTable> </changeSet>
</databaseChangeLog>
![Page 49: Modularized Persistence - B Zsoldos](https://reader034.fdocuments.us/reader034/viewer/2022052508/5597968a1a28ab73538b45d6/html5/thumbnails/49.jpg)
Full Text Search
● Analize the different solutions– H2 → Lucene
– MySQL → Sphinx
– PostgreSQL → TSearch2
– SQL Server →???– Oracle → Oracle Text
● Create a common API for Querydsl based query extension
● Create a module for each database engine
![Page 50: Modularized Persistence - B Zsoldos](https://reader034.fdocuments.us/reader034/viewer/2022052508/5597968a1a28ab73538b45d6/html5/thumbnails/50.jpg)
Querydsl: http://www.querydsl.com/
Liquibase: http://www.liquibase.org/
Liquibase OSGi Bundle: https://github.com/everit-org/osgi-liquibase-bundle
LQMG: https://github.com/everit-org/osgi-lqmg
LQMG Maven Plugin: http://www.everit.org/lqmg-maven-plugin/
Liquibase DataSource: https://github.com/everit-org/osgi-liquibase-datasource
Querydsl SQLTemplates Component: https://github.com/everit-org/osgi-querydsl-templates
Querydsl Configuration Component: https://github.com/everit-org/osgi-querydsl-configuration
Querydsl Support Component: https://github.com/everit-org/osgi-querydsl-support
Cookbook chapter: http://cookbook.everit.org/persistence/index.html
Download from Maven central
Resource: https://github.com/everit-org/resource-ri
Authorization: https://github.com/everit-org/authorization-ri
Authentication: https://github.com/everit-org/authentication-simple
Property Manager: https://github.com/everit-org/property-manager-ri
Twitter: @EveritOrg