An Multiple Regression Analysis Based

Color Transform Between Objects

Speaker ： Chen-Chung Liu

11

OutlineIntroduction

The proposed algorithm◦Color Objects Extraction Algorithm Using

Multiple Thresholds (COEMT)

◦Color Transform Using Multiple Regression

Analysis (MRA)

Conclusions2

1. Introduction(1/3)Art purpose

3

1. Introduction (2/3)Image analysis (details increasing)

4

1. Introduction (3/3)Image analysis (image simplify)

5

The proposed algorithm

6

Figure 1. The flow chart of the proposed color transformation algorithm.

2.1.Color Objects Extraction(1/17)

Figure 2. Color objects extraction algorithm flow chart.

7

IRGBI DSROI CSSC

RGB2HSI EOAFFSSC OER

IRGBI: Input RGB Color ImageDSROI: Draw Symbols on the Region of InterestCSSC: Capture and Store the Symbols’ CoordinatesRGB2HSI: Transform Image Color Space from RGB to HSISSC: Set Searching Coordinates on HSI imageEOAFF: Extract Object Using Adaptive Forecasting FiltersOER: Output the Extracted Result

Figure 3. Pixels values distribution on different planes.Figure 3. Pixels values distribution on different planes.

2.1.Color Objects Extraction (2/17)

8

(a) original RGB image (b) on R plane (c) on G plane (d) on B plane

(e) on H plane (f) on S plane (g) on I plane (h) on |S-I| plane

2.1.Color Objects Extraction (3/17)

Figure 4. Intensity versus RGB and saturation versus RGB.

9

2.1.Color Objects Extraction (4/17)

Figure 5. The flow chart of EOAFF on HSI domain.

10

SNSS

CAFFSEC CATV

CS on H&I

BSE on S&I

MCSR CR OER

IIMTrue

False

IHFW ,

ISFW ,

tS

tH tI

IIM: Input HSI Image and Markers SNSS: Set New Searching SeedsSEC: Search by Eight-ConnectivityCAFF: Create the Adaptive Forecasting FiltersCATV: Calculate the Adaptive Threshold Vectors

CE on H&I: Color Search on H&I Planes with SLCCABSE on S&I: Bright and Shadow pixels Extraction on S&I Planes with SLCCAMCSR: Merging of CS result and BDE ResultCR: Check Result Whether Have Any SeedOER: Output the Extracted Result

2.1.Color Objects Extraction(5/17)

11

I(1)-I(0)*2(3)I'

S(1) -S(0)*2 (3)S'

H(1)}/2 H(0) {(3)H'

I(5) -I(0)*2 (7)I'

S(5) -S(0)*2 (7)S'

H(5)}/2 H(0) {(7)H'

2.1.Color Objects Extraction(6/17)

12

I(0)}/2 I(1) {(5)I'

S(0)}/2 S(1) {(5)S'

H(0)}/2 H(1) {(5)H'

I(5)}/2 {I(0) (1)I'

S(5)}/2 {S(0) (1)S'

}/2 H(5) H(0) {(1)H'

2.1.Color Objects Extraction(7/17)

13

I(1) -I(5)*2 (2)I'

S(1) -S(5)*2 (2)S'

H(5)}/2 H(1) {(2)H'

I(5) -I(0)*2 (1)I'

S(5) -S(0)*2 (1)S'

H(8)}/3 H(5) H(0) {(1)H'

2.1.Color Objects Extraction(8/17)

14

I(2)}/2 I(1) {(5)I'

S(2)}/2 {S(1) (5)S'

H(2)}/2 {H(1) (5)H'

,81,i I(0), (i)I'

,81,i S(0), (i)S'

,81,i H(0), (i)H'

Filter’s thresholds of hue , saturation , and intensity

2.1.Color Objects Extraction(9/17)

15

otherwiseHiHabs

HHiHabsifH

HHiHabsifH

H

i

mim

MiM

t

),)]0()('[max(

))]0()('[max(,

))]0()('[max(,

81

81

81

otherwiseSiSabs

SSiSabsifS

SSiSabsifS

S

i

mim

MiM

t

),)]0()('[max(

))]0()('[max(,

))]0()('[max(,

81

81

81

otherwiseIiIabs

IIiIabsifI

IIiIabsifI

I

i

mim

MiM

t

),)]0()('[max(

))]0()('[max(,

))]0()('[max(,

81

81

81

the global empirical constants mH =0.008, MH =0.032, mS =0.024, MS =0.048, mI =0.024, and

MI =0.048.

2.1.Color Objects Extraction(10/17)

Figure 6. An example of the proposed adaptive forecasting filter‘s working.

16

2.1.Color Objects Extraction(11/17)

Figure 7. An example of the proposed scheme.

17

original image union result

CS result BSE result

2.1.Color Objects Extraction(12/17)

18

Figure 8. Test image: Pink hat.

original imagewith seeds

DTS in RGB DTS in HSI proposed scheme

C. C. Liu and G. N. Hu, Color Objects Extraction Scheme Using Dynamic Thresholds (DTS), 2009 Workshop on Consumer Electronics (WCE2009), pp. 1130-1138, 2009.

2.1.Color Objects Extraction(13/17)

19

Figure 9. Test image: Flowers.

original imagewith seeds

DTS in RGB DTS in HSI proposed scheme

2.1.Color Objects Extraction(14/17)

20

Figure 10. Test image: Pottery.

original image with seeds DTS in RGB

DTS in HSI proposed scheme

2.1.Color Objects Extraction(15/17)

21

Figure 11. Test image: Cup set.

original image with seeds DTS in RGB

DTS in HSI proposed scheme

2.1.Color Objects Extraction(16/17)

22

Figure 12. Test image: Sun flower.

original image with seeds DTS in RGB

DTS in HSI proposed scheme

Image Extraction scheme ME RFAE EMM NU MHD Accuracy

Pink hat(325×415)

DTS on RGB 0.0355 0.2549 0.8319 0.8696 4.4061 0.9645

DTS on HSI 0.0079 0.0537 0.0975 0.8629 0.696 0.9921Proposed 0.0079 0.0397 0.0949 0.8615 0.0625 0.9921

Flowers(172×222)

DTS on RGB 0.3726 0.8586 0.9556 4.4272 11.737

1 0.6274DTS on HSI 0.0514 0.1119 0.1835 1.1290 0.1848 0.9486Proposed 0.0186 0.0085 0.0664 0.6315 0.0372 0.9814

Pottery(350×251)

DTS on RGB 0.3140 0.8305 0.9012 1.5588 16.196

0 0.6860DTS on HSI 0.0841 0.2267 0.4983 0.9522 0.4927 0.9159Proposed 0.0160 0.0263 0.0461 0.6720 0.0372 0.9840

Cup set(599×399)

DTS on RGB 0.0621 0.1490 0.6542 1.2389 0.5322 0.9379

DTS on HSI 0.0286 0.0677 0.1874 0.7566 0.2494 0.9714Proposed 0.0066 0.0059 0.0417 0.6214 0.0124 0.9934

Sunflower(768×1024

)

DTS on RGB 0.3716 0.8323 0.9784 1.1597 95.459

6 0.6284

DTS on HSI 0.3795 0.8181 0.9650 1.1492 93.8753 0.6205

Proposed 0.0434 0.0862 0.4841 0.5430 0.0121 0.9566

2.1.Color Objects Extraction(17/17)

23

Table 1. Comparisons of extraction results

Multiple Regression Analysis (1/5)

For data of ordered pairs

We want to predict y from x by finding a function that fits the data as closely as possible.

2.2. MRA_based Color Transform(1/20)

24

),(),...,,(),,( 2211 nn yxyxyx

)(xHy

Multiple Regression Analysis (2/5)MRA is used to find a polynomial function of degree , as the predicting function, that has the minimum of the sum of squares of the errors(SSE) between the predicted values of y and the observed values for all of the n data points .

2.2. MRA_based Color Transform(2/20)

25

k kk xxxy ...2

210

iy),(),...,,(),,( 2211 nn yxyxyx

Multiple Regression Analysis (3/5)The values of , , ,…,and that minimize

are obtained by setting the first partial derivatives ,

,…, andequal to zero.

2.2. MRA_based Color Transform(3/20)

26

0 1 2k

2

1

221010 )]...([),...,,(

n

i

kikiiik xxxySSE

),...,,( 100

kSSE

),...,,( 101

kSSE

),...,( 10 kk

SSE

Multiple Regression Analysis (4/5)Solving the resulting simultaneous linear system of the so-called normal equations:

2.2. MRA_based Color Transform(4/20)

27

n

ii

n

i

n

i

n

i

kikii yxxxn

11 1 1

2210 ...

n

iii

n

i

n

i

n

i

kikii

n

ii yxxxxx

11 1 1

132

21

10 ...

n

ii

ki

n

i

n

i

n

i

kik

ki

ki

n

i

ki yxxxxx

11 1 1

222

11

10 ...

Multiple Regression Analysis (5/5)The matrix form solution be

where

2.2. MRA_based Color Transform(5/20)

28

YXXX TT

k

1

2

1

0

:

,

...1

:::::::

...1

...1

...1

2

3233

2222

1211

knnn

k

k

k

xxx

xxx

xxx

xxx

X .

.

.

.2

1

ny

y

y

Y

2.2. MRA_based Color Transform(6/20)

29Figure 13. Target object .

2.2. MRA_based Color Transform(7/20)

30Figure 14. Source object .

Best fitting functions

2.2. MRA_based Color Transform(8/20)

31

Red Green Blue

Figure 15. The curves of degree1, 5, and 9 best fitting functions.

2.2. MRA_based Color Transform(9/20)

32

Figure 16. The color transfer results corresponding to the variation in the degree of best fitting polynomials.

2.2. MRA_based Color Transform(10/20)

33

L* a* b*

Figure 17. The box-plots of L*, a*, and b* for the target, source, and color transferred objects in Figure 11.

CIELAB L* a* b*

MEAN STD MEAN STD MEAN STD

Dress150.430

4

51.4691

7

106.423

6

8.97858

9

117.770

9

5.47792

5

chrysanthemum215.671

1

39.2809

4

124.722

2

7.94416

2

200.105

7

26.2307

9

Degree 1217.829

2

31.3322

5

123.691

6

8.80164

8

195.963

1

28.1982

3

Degree 2220.721

5

36.3131

7

125.550

2

10.5864

8

196.376

9

30.1589

7

Degree 3 220.22735.1729

5

127.664

9

10.9081

5

196.602

9

29.9496

7

Degree 4220.203

4

33.8926

9

126.890

6

11.9964

5

196.317

1

30.0778

9

Degree 5220.272

733.9665

126.950

8

11.8242

7

196.553

6

29.7591

8

Degree 6220.278

1

33.7996

2

126.995

1

11.8225

9

196.376

6

30.0007

5

Degree 7220.398

833.6955

127.098

5

11.9772

4

196.568

1

29.8268

3

Degree 8221.886

7

30.2697

1

126.462

9

11.0972

2

197.157

7

29.2396

2

Degree 9222.843

7

29.3369

2

125.709

7

11.8568

3

193.991

3

29.5341

3

2.2. MRA_based Color Transform(11/20)

34

Table 2. The measurement metrics for the target, source and color transferred objects in Figure 17 (1/2)

CIELAB C* H* E*

MEAN STD MEAN STD MEAN STD

Dress 158.7859.69240

1

42.0409

8

1.43258

2

221.787

1

37.3548

2

chrysanthemum236.502

4

20.3878

7

32.3032

94.79458

321.949

2

27.4735

3

Degree 1232.656

321.0893

32.7115

3

5.44170

1

320.265

6

20.8540

4

Degree 2234.177

7

22.5643

5

33.0888

3

5.88526

6

323.778

1

23.5091

2

Degree 3235.451

7

23.0103

8

33.4690

8

5.70499

3

324.393

3

21.7474

3

Degree 4234.875

3

22.8860

2

33.3507

7

5.92635

1

323.891

4

20.5603

4

Degree 5235.065

1

22.7958

4

33.3198

4

5.80953

1

324.087

9

20.3967

5

Degree 6234.962

522.8892

33.3619

5

5.87509

8

323.999

9

20.4977

8

Degree 7235.160

7

22.9257

6

33.3475

7

5.81623

5

324.225

4

20.3704

8

Degree 8235.195

9

22.9131

533.1205

5.52175

2

325.013

5

18.9275

9

Degree 9232.150

7

23.5463

7

33.3893

8

5.61041

9

323.509

7

17.6013

5

2.2. MRA_based Color Transform(12/20)

35

Table 2. The measurement metrics for the target, source and color transferred objects in Figure 17 (2/2)

The target RGB color image is a girl in a blue dress (350×350 pixels).

The source color images with different sizes.

2.2. MRA_based Color Transform(13/20)

36

Image Size (pixel)

Blue Roses 528×458

Wool Hat 450×377

Potted Plant 750×1000

Amber 399×354

Carnation Flower 640×480

The extraction procedure lasted between 3 and 25 seconds, and the color transferring procedure lasted about 0.03 seconds.

2.2. MRA_based Color Transform(14/20)

37

2.2. MRA_based Color Transform(15/20)

38

Figure 20. Examples of color transferring between objects with the proposed multiple regression analysis algorithm (1/2).

2.2. MRA_based Color Transform(16/20)

39

Figure 21. Examples of color transferring between objects with the proposed multiple regression analysis algorithm (2/2).

Performance measures function:

2.2. MRA_based Color Transform(17/20)

40

22 *)(*)(* baC

222 *)(*)(*)(* baLE

/*))/*(2arctan180(* abH

*}*,*,*,*,*,{,/)(1

EHCbaLXNjXXN

j

st XXX

)/|(|100(%) sXXX

2.2. MRA_based Color Transform(18/20)

41

CIELABL* a* b*

MEAN STD MEAN STD MEAN STD

Blue dress150.430

451.4691

7106.423

68.97858

9117.770

95.47792

5

Blue roses144.574

341.9283

9134.401

713.1304

274.8278

16.13991

Wool hat156.117

740.6513

8106.235

613.2233

6162.011

222.8879

3

Potted plant145.215

937.9356

399.2557

99.78578

7171.883

414.5488

3

Amber145.216

336.4920

2108.842

721.2677

3173.322

914.3342

9

Carnation 147.322

940.4143

6150.093

433.0741

80.18154

19.15783

CIELABC* H* E*

MEAN STD MEAN STD MEAN STD

Blue dress 158.7859.69240

142.0409

81.43258

2221.787

137.3548

2

Blue roses154.988

78.62681

460.8049

36.87635

7214.632

526.2399

Wool hat195.139

912.3608

533.6504

7.001189

252.3068

24.4726

Potted plant199.036

59.35587

230.1465

94.36541

1248.277

624.2501

8

Amber205.627

616.2058

732.0184

5.473507

253.6519

25.00947

Carnation 171.544

631.4717

761.4555

27.18140

1229.262

934.5528

2

Table 3. The measurement metrics for the target and source objects in Figures 20,21

2.2. MRA_based Color Transform(19/20)

42

CIELABL* a* b*

MEAN STD MEAN STD MEAN STD

Blue roses163.318

555.6394

9128.507

417.0255

790.4183

731.5838

Wool hat188.812

639.3171

1126.217

512.7337

7130.383

311.1928

4

Potted plant162.481

453.2363

1102.419

123.9186

7162.492

421.2341

5

Amber168.295

352.2510

7142.565

116.4569

6164.566

21.28246

Carnation 169.981

447.7605

6185.628

633.0194

6102.968

516.2365

4

CIELABC* H* E*

MEAN STD MEAN STD MEAN STD

Blue roses160.051

818.9847

655.3171

510.5985

1231.989

343.8929

2

Wool hat181.843

512.2898

344.0244

93.45239

7264.100

625.7869

7

Potted plant194.418

810.8495

232.2716

68.74632

7257.289

330.8681

3

Amber218.257

622.2291

240.9812

94.04682

1280.646

420.5305

4

Carnation 214.366

121.4830

960.2257

88.36698

2277.923

118.6482

1

Table 4. The measurement metrics for the color transferred target objects in Figures 20, 21

CIELAB ΔL* ΔL*(%) Δa* Δa*(%) Δb* Δb*(%)Blue roses 18.7442

211.4770

95.89427 4.58671

515.5905

717.2427

Wool hat 32.69487

17.31605

19.98188

15.83131

31.62791

24.25764

Potted plant 17.26554

10.62616

3.163266

3.088552

9.391059

5.779385

Amber 23.07898

13.71338

33.7224 23.65404

8.756862

5.321186

Carnation 22.6585 13.32999

35.5352 19.14317

22.78695

22.13002

2.2. MRA_based Color Transform(20/20)

43

CIELAB ΔC* ΔC*(%) ΔH* ΔH*(%) ΔE ΔE*(%)Blue roses 5.06312

13.16342

65.48777

89.92057

317.3568

47.48174

3Wool hat 13.2963

97.31199

510.3740

923.5643

511.7937

54.46563

Potted plant 4.617675

2.375117

2.125071

6.584944

9.011727

3.502566

Amber 12.63007

5.786773

8.962891

21.87069

26.99445

9.618671

Carnation 42.82152

19.97588

1.229737

2.041878

48.66021

17.50852

Table 5. The absolute difference in measurement metrics of the transferred target-object from the source object in Figures 20 and 21

Simple, effective and accurate in color transferring between objects.

Details of target object can be changed by the color complexity of source object.

Time consumption is independent of the number of bins selected and the degree of regression.

Dynamic ranges of colors of objects don’t have any restriction.

Conclusions

44

45

Thank YouQuestions and Comments