3D models in retail at pervasive scale 3D Commerce: The ...

© The Khronos® Group Inc. 2021 - This work is licensed under a Creative Commons Attribution 4.0 International License © The Khronos® Group Inc. 2021 - Page 1This work is licensed under a Creative Commons Attribution 4.0 International License

3D Commerce: The art and business of using 3D models in retail at pervasive scale

Nathaniel Hunter - Dreamview Studios, The Khronos GroupMax Limper – DGG, The Khronos GroupBrian Freisinger - Dreamview Studios, The Khronos GroupJatinder Kukreja - SuperDNA 3D Lab, The Khronos GroupPaweł Nikiel – CGTrader, The Khronos Group

https://creativecommons.org/licenses/by/4.0/

© The Khronos® Group Inc. 2021 - This work is licensed under a Creative Commons Attribution 4.0 International License

Today’s Speakers

Nathaniel HunterDreamview

Jatinder KukrejaSuperDNA 3D Lab

Paweł NikielCGTrader

Brian FreisingerDreamview

Max LimperDGG


Nathaniel Hunter• Co-Chair of 3D Commerce Asset Creation TSG

• 30 years in content creation spanning film, TV, commercials, games, products, and internet technology

• Chief Executive Officer at DreamView, Inc.

- B2B online platform serving retailers, suppliers, manufacturers and brand owners

- Scalable and standardized management, creation and distribution of real-time and photo-real product content

pawelnikiel.com@pawel_nikiel

cgtrader.com

http://www.pawelnikiel.com/

https://twitter.com/pawel_nikiel

http://www.cgtrader.com/


Max Limper• Co-Chair of 3D Commerce Asset Creation TSG

• glTF spec contributor (binary glTF, 2.0 PBR model)

• PhD in CS on 3D data optimization(Fraunhofer / TU Darmstadt)

• CTO & Co-Founder at DGG, creating RapidCompact- Commercial 3D asset optimization solution

serving leading retailers & 3D studios

- Tech & workflows aligned with standards(glTF, USDZ, Asset Creation Guidelines)

- Additional open-source scripts on GitHub max-limper.de@mlimper_cg

rapidcompact.com

http://www.max-limper.de/

https://twitter.com/mlimper_cg

https://rapidcompact.com/


Brian Freisinger

• 30 years in content creation spanning film, TV, commercials, games, products, and internet technology

• CG Supervisor at DreamView, Inc.

- B2B online platform serving retailers, suppliers, manufacturers and brand owners

- Scalable and standardized management, creation and distribution of real-time and photo-real product content


Jatinder Kukreja

[email protected]

mailto:[email protected]


Paweł Nikiel• Contributor to 3D Commerce WG

• Over 10 years of experience in 3D art,VR & AR development, pipeline & automation,data visualization

• Everything interactive (WebGL, game engines etc.)

• 3D Automation Lead at CGTrader

- Batch model processing and validation

- Tools development

- Automated assets generation

- Interactive assets developmentpawelnikiel.com

@pawel_nikielcgtrader.com

http://www.pawelnikiel.com/

https://twitter.com/pawel_nikiel

http://www.cgtrader.com/


Impact of 3D in Retail

“...46 percent of retailers planned to deploy either AR or VR solutions to meet customer service experience requirements….”

“...the impact of AR or VR in retail can be transformative...”

“...retailers can use AR as an extension of the brand experience to engage customers in immersive environments and drive revenue…Additionally, AR can be used outside the store

after a sale to increase customer satisfaction and improve loyalty...”

April 2019 Gartner press release

https://www.gartner.com/en/newsroom/press-releases/2019-04-01-gartner-says-100-million-consumers-will-shop-in-augme


Let’s Think About “Search” as a Use CaseE.g. Google Search now enables 3D and AR experience directly from Search results

View in 3DRetailer’s Content

View in AR

Retailer’s Website

Buy Now

Link Retailer’s Content Link


View in 3D

Retailer’s Content

View in AR


Buy NowRetailer’s Content


View in AR




View in AR



…. Across different platformsInconsistent visual quality across experiences


Where Does 3D Content Come From ?

View in 3D


View in AR




View in AR




View in AR



3D Content Production


3D Content ProductionOne example of the content production process

Manufacturer RetailerEnd-User

Platform

Content Creator

3D Tools

Uses

Uses

Distributes via Publishes on

Uses View in 3D

Search


3D Commerce - The Promise

IKEA catalog uses augmented reality to give a virtual preview of furniture in a room – August 2013!

Early Experience ShowsIncreased customer engagement!

Strengthened brand loyalty!Deeper product understanding!

More online sales!Fewer returns!

So why is 3D Commerce taking so long to become pervasive?

=$$$!


3D Commerce Initiative – the Opportunity

View in 3D

table lamp

table lamp

Enable 3D Commerce to achieve deployment at industrial scale Industry cooperation between technology and retail leaders

Interoperability standards, guidelines and Certification Programs


State of the Industry Before 3D Commerce

Products don’t come with 3D data – and I can’t physically scan

them all fast enough!

The green couch looks blue on some devices – lots of product returns

are expensive!

CAD tools don’t let me easily generate

the data I need for E Commerce!

Many 3D products on my ecommerce

web-site first appear upside down! I have to

hand-tune 1000s of models!

I wish I had high quality, realistic 3D models for virtual

promotional photoshoots!

Everyone defines their product data for sizes and colors differently

– nothing is consistent!

I need the materials in my 3D models to look

completely realistic!

Complex retail pipeline with hundreds of companies and thousands of productsMany friction points - tooling, technical and commercial


Khronos 3D Commerce Mission

Create specifications and guidelines to align the 3D asset workflow from product design through manufacturing, through

each stage of retail to end-user delivery platforms


Khronos 3D Commerce Mission


Khronos Active Initiatives

3D GraphicsDesktop, Mobile

and Web

3D Assets Authoring

and Delivery

Portable XRAugmented and Virtual Reality

Parallel ComputationVision, Inferencing,Machine Learning

Safety Critical APIs


Diverse Technology and E-commerce Companies Working Together


Khronos 3D Commerce Goals

Visual realism and consistency no matter where

the model is displayed



Product display configurability with

consistency and authenticity



Structured metadata for product management and configurability of viewing



Reduce production, distribution and marketing costs

Guidelines for tools and product designers to create assets with consistent data to be used through the 3D Commerce pipeline



Reduce production, distribution and marketing costs

Visual realism and consistency no matter where

the model is displayed

Product display configurability with

consistency and authenticity

Structured metadata for product management and configurability of viewing

Guidelines for tools and product designers to create assets with consistent data to be used through the 3D Commerce pipeline


Images: OTTO, IKEA, Shopify

IKEA Place

OTTO yourhome

AR in Shopify

AR in E-Commerce Today!

Which of these is the “real one”? Source: Shopifyhttps://www.youtube.com/watch?v=ajF5Rasyq3o&t=428s

• Technology is there!

https://www.youtube.com/watch?v=ajF5Rasyq3o&t=428s


?3D assets for real-time apps:AR + Web technology

Images: IKEA

Offline CGI

Large Files

Complex Materials

Millions of 3D AssetsReal-T

ime Rendering

Compact Files

Real-Time PBR Materia

ls

The “Offline / Real-Time Content Gap”


Optimization “recipes” for various sources

3D-Scanned Assets• Simplification

• Material tweaking & texture baking

CAD data• Tessellation & geometry cleanup

• Rule-based, automated materials assignment

Assets from 3D modeling (DCC tools)• Simplification

• Material conversion & texture baking

[2] https://rapidcompact.com/case-study-markgraph/[1] https://twitter.com/nebulousflynn/status/1372589608165384201

[3] https://rapidcompact.com/case-study-otto/

1

2

3

https://rapidcompact.com/case-study-markgraph/

https://twitter.com/nebulousflynn/status/1372589608165384201

https://rapidcompact.com/case-study-otto/


Scalable 3D Today: Two Secret Ingredients

Automate 3D optimization tasks

• Use “recipes” for different archetypes of 3D assets / sources

• Automate creation of variants for publishing targets,without the artist having to care about it (“YouTube approach”)

• Enable easy upstream modifications (easy to re-run where necessary)

Use Web standards, such as glTF

• High compatibility

• Support across different DCC toolchains & viewers

• Large ecosystem (including many free tools!) available


3D Commerce Deliverables

Extensions

PBR MaterialsTransmission

ClearcoatSheen

RefractionSpecular Color

Color AttenuationVolumetric Properties

Consistent Asset Creation

3D Commerce Viewer Certification

Program

Free guidelines for tools and product designers to create high-quality

real-time 3D models for e-commerce

Material VariantsEfficient

transmission of many e-commerce

products

Open-Source 3D Viewers

Retailer 3D Viewers

Non-retailer 3D Viewers

Non-Retailer Web use cases

Social, Ads, Search …

Retailer WebStore Fronts

Brands /Manufacturers

Future ExperiencesWeb, Apps,

Mobile, AR ...

Viewers certified to accurately display 3D productsConsistent deployment in a wide variety of end-user

experiences: Web, social media, ad platforms, apps etc.

End User Experiences On the Web and in Apps

Desktops and mobile RenderingTechnologies

How to use KTX 2.0 supercompressed textures

for minimized glTF download size at high quality

MetadataSupports for XMP Extensible

Metadata Platform ISO 16684-1 to hold consistent data for

product discovery, dimensions, how a virtual product interacts

with the physical world, defining variants, etc.

Certification to go beyond testing rendered pixels and

achieve ecosystem consistency around tone mapping and

display calibration

Open-source asset checker tool in design to guide creation and assess compliance of reliable

e-commerce 3D assets

glTF Roadmap discussions underway: more PBR

materials, realistic cloth and apparel, constraints for

simulation etc… Metadata conventions

Next Steps


Takeaways

▪ There are a lot of 3D scans, CGI & CAD data sets to be made “real-time ready”

▪ Automating optimization is key!

▪ Relying on 3D standards is key!


Example Product


Asset Creation, QC & QA


Color Variations


Photo-real use cases


Example Product


Asset Creation, QC & QA


Variations


An Example Process


DEMONSTRATION

Showcasing real world problems and solutions with 3D content for:- 3d Commerce Guidelines based modeling and texturing- Photo-real products- Augmented Reality


Benefits of automated processing

DGG RapidCompact Customer Case: Preparing 5,300 CGI assets for the Web/AR (glTF & USDZ)(3D models of furniture products)

Estimated manual effort: 2,650+ Hours(30 mins to 1h manual work per asset)

Automated optimization: 54.7 Hours ● Polygon reduction● Texture baking (incl. AO)● Scene graph flattening● Draw call reduction● Compression (Draco) & glTF export


Automating optimization for real-time applications

Original Image: Khronos Asset Creation Guidelines v1.0https://github.com/KhronosGroup/3DC-Asset-Creation/blob/main/asset-creation-guidelines/full-version/sec99_PublishingTargets/PublishingTargets.md

https://github.com/KhronosGroup/3DC-Asset-Creation/blob/main/asset-creation-guidelines/full-version/sec99_PublishingTargets/PublishingTargets.md


Case study - DreamView & DGG

Read the full story here:https://rapidcompact.com/siggraph2021-demo-production-to-real-time/

CGI Asset

Real-Time Asset

Try it outyourself!

https://rapidcompact.com/siggraph2021-demo-production-to-real-time/


Case study - DreamView & DGG


From V-Ray to Physical Material

Polygons: 500K to 20KDraw Calls: 63 to 6File Size: 180MB (FBX) to 9MB (glTF)Processing Time: 38s

Real-Time Asset(“High Resolution” Version)9MB Optimized glTF



Jack Asteroid in AR!


Converting Jack Asteroid into an AR versionAnimation baking and FBX export, compression with RapidCompact to glTF & USDZ



Jack Asteroid in AR!


Try it out yourself!

http://www.youtube.com/watch?v=3OBh_-qkBAU



Challenges & Opportunities

Jatinder @ SuperDNA 3D Lab


The future of 3D Commerce

▪ Enable users to configure their own products.

▪ Enable users to design their own products.

▪ Let the user play with your product - make it interactive!


Get Involved in Khronos 3D Commerce!3D Commerce BOF Tomorrow with lots more details

3D Commerce: The Art and Business of Using 3D Models in Retail at Pervasive ScaleTuesday, August 10 from 3-4:30pm PDT

Join Khronos and the 3D Commerce Working GroupWe welcome both commerce and technology companies

Influence and gain early access to extensions, tools, programs

Or join the 3D Commerce Advisory ForumProvide requirements and feedback to the working group

Free of charge in any interested companyhttps://www.khronos.org/3dcommerce/

[email protected]

Tuesday, August 10 from 3-4:30pm PDT

https://s2021.siggraph.org/session/?sess=sess424

https://www.khronos.org/3dcommerce/

mailto:[email protected]


Khronos Group BOF Series at SIGGRAPH

• OpenXR: On an HMD Near You Now!- Tuesday, August 10 from 1-2pm PDT

• 3D Commerce: The Art and Business of Using 3D Models in Retail at Pervasive Scale- Tuesday, August 10 from 3-4:30pm PDT

• An Exploration of Pervasive 3D in Geospatial with glTF- Wednesday, August 11 from 10-11am PDT

• Vulkan Without Boundaries: Case Studies from the Real World- Wednesday, August 11 from 1-3pm PDT

• Hands on with glTF: Making 3D pervasive- Thursday, August 12 from 10-11:30am PDT

www.khronos.org/events/2021-siggraph

https://s2021.siggraph.org/presentation/?id=bof_130&sess=sess423



https://s2021.siggraph.org/presentation/?sess=sess517&id=bof_147#038;id=bof_147

https://s2021.siggraph.org/presentation/?sess=sess426&id=bof_133#038;id=bof_133

http://www.khronos.org/events/2021-siggraph

© The Khronos® Group Inc. 2021 - This work is licensed under a Creative Commons Attribution 4.0 International License © The Khronos® Group Inc. 2021 - Page 50This work is licensed under a Creative Commons Attribution 4.0 International License

https://creativecommons.org/licenses/by/4.0/

3D models in retail at pervasive scale 3D Commerce: The ...

Documents

Transcript of 3D models in retail at pervasive scale 3D Commerce: The ...