Post on 03-Feb-2022
© 2007 Perforce Software 3
www.snopes.com
• Research validity of outrageous claims• for those emails that "just don't seem right"
• Example:• Claim: {eTailer} suspending account until updated• Status: False
• Urban legends, common fallacies, etc.• Advertisements aplenty
© 2007 Perforce Software 4
Objectives
• Explore:• metadata locking• filesystem cache• exclusionary entries• embedded wildcards
• Expose misconceptions• Reveal facts not (yet) common knowledge• Discuss some Perforce Server internals
© 2007 Perforce Software 5
Perforce Metadata
• Claim: Perforce Server uses metadatastored in the db.* files
• Status: ?
© 2007 Perforce Software 6
Perforce Metadata
• Claim: Perforce Server uses metadatastored in the db.* files
• Status: True
© 2007 Perforce Software 7
Perforce Metadata
• Claim: Perforce Server uses metadatastored in the db.* files
• Status: True• For example:
• db.rev, db.change, db.integed, db.have, etc.• Only used by server, not clients• Database schema located at:
www.perforce.com/perforce/doc.current/schema/index.html
• also applicable to checkpoint and journal entries
© 2007 Perforce Software 8
Metadata and Concurrency
• Multiple commands processed simultaneously• Each command must use:
• physically consistent metadata• B-trees in each db.* file must be structurally correct
• e.g. writer not splitting pages a reader is traversing
• logically consistent metadata• db.* files correctly contain related data
• e.g. db.rev* entries exist for submitted changelist
• Synchronization required for metadata access
© 2007 Perforce Software 9
Metadata Locking
• Claim: There is a single global lock controllingall metadata access
• Status: ?
© 2007 Perforce Software 10
Metadata Locking
• Claim: There is a single global lock controllingall metadata access
• Status: ?• Claim: The single global lock is an exclusive lock,
serializing all metadata access• Status: ?
© 2007 Perforce Software 11
Metadata Locking
• Claim: There is a single global lock controllingall metadata access
• Status: False• Claim: The single global lock is an exclusive lock,
serializing all metadata access• Status: False
© 2007 Perforce Software 12
Metadata Locking
• Claim: There is a single global lock controllingall metadata access
• Status: False• Claim: The single global lock is an exclusive lock,
serializing all metadata access• Status: False• If true, these would severely limit concurrency
© 2007 Perforce Software 13
Metadata Locking
• Each command locks only the db.* files it needs• A lock on a db.* file is either shared or exclusive
• shared lock• allows access by others using a shared lock• used when might read but won’t write to the db.* file
• exclusive lock• denies access by others• used when might write to the db.* file
© 2007 Perforce Software 14
Observe Metadata Locking
• Use OS utilities to observe locking behavior• e.g. strace on Linux, truss on FreeBSD and
Solaris, and FileMon on Windows• For example (on Linux):
$ strace -etrace=open,flock,close p4d -f -p 50620(system calls for server startup)Perforce Server starting...
• -f (single-threaded server) to simplify tracing• trace only open(), flock(), and close() calls
© 2007 Perforce Software 15
Observe Metadata Locking
• From another session:$ p4 -p 50620 sync //depot/main/... //depot/branch1/...//depot/main/file1#1 - added as /p4client/depot/main/file1//depot/main/file2#1 - added as /p4client/depot/main/file2//depot/main/file3#1 - added as /p4client/depot/main/file3//depot/branch1/file1#1 - added as /p4client/depot/branch1/file1//depot/branch1/file2#1 - added as /p4client/depot/branch1/file2//depot/branch1/file3#1 - added as /p4client/depot/branch1/file3
• two arguments• head revisions implied
• six files synced
© 2007 Perforce Software 16
Command Startup Phaseopen("journal", O_WRONLY|O_APPEND|O_CREAT|O_LARGEFILE, 0666) = 5open("db.depot", O_RDWR|O_CREAT|O_LARGEFILE, 0666) = 6flock(6, LOCK_SH) = 0flock(6, LOCK_UN) = 0open("license", O_RDONLY|O_LARGEFILE) = 7close(7) = 0open("db.group", O_RDWR|O_CREAT|O_LARGEFILE, 0666) = 7flock(7, LOCK_SH) = 0flock(7, LOCK_UN) = 0open("db.protect", O_RDWR|O_CREAT|O_LARGEFILE, 0666) = 8flock(8, LOCK_SH) = 0flock(8, LOCK_UN) = 0open("db.user", O_RDWR|O_CREAT|O_LARGEFILE, 0666) = 9flock(9, LOCK_EX) (LOCK_SH in 2007.2!) = 0flock(9, LOCK_UN) = 0open("db.domain", O_RDWR|O_CREAT|O_LARGEFILE, 0666) = 10flock(10, LOCK_EX) (LOCK_SH in 2007.2!) = 0flock(10, LOCK_UN) = 0open("db.view", O_RDWR|O_CREAT|O_LARGEFILE, 0666) = 11flock(11, LOCK_SH) = 0flock(11, LOCK_UN) = 0
© 2007 Perforce Software 17
Compute Phase for First Argument
open("db.have", O_RDWR|O_CREAT|O_LARGEFILE, 0666) = 12flock(12, LOCK_SH) = 0open("db.resolve", O_RDWR|O_CREAT|O_LARGEFILE, 0666) = 13flock(13, LOCK_SH) = 0open("db.revdx", O_RDWR|O_CREAT|O_LARGEFILE, 0666) = 14flock(14, LOCK_SH) = 0open("db.revhx", O_RDWR|O_CREAT|O_LARGEFILE, 0666) = 15flock(15, LOCK_SH) = 0open("db.working", O_RDWR|O_CREAT|O_LARGEFILE, 0666) = 16flock(16, LOCK_SH) = 0flock(16, LOCK_UN) = 0flock(15, LOCK_UN) = 0flock(14, LOCK_UN) = 0flock(12, LOCK_UN) = 0flock(13, LOCK_UN) = 0
• computes dataset passed on to next phase• db.revhx and db.revdx locked instead of db.rev
• head revisions implied by argument
© 2007 Perforce Software 18
Execution Phase for First Argument
open("depot/main/file1,v", O_RDONLY|O_LARGEFILE) = 17close(17) = 0open("depot/main/file2,v", O_RDONLY|O_LARGEFILE) = 17close(17) = 0open("depot/main/file3,v", O_RDONLY|O_LARGEFILE) = 17close(17) = 0
• file content retrieved and sent to client• no db.* file locks held
• for this example, db.have is not updatedduring this execution phase
© 2007 Perforce Software 19
Compute Phase for Second Argument
flock(12, LOCK_SH) = 0flock(13, LOCK_SH) = 0flock(14, LOCK_SH) = 0flock(15, LOCK_SH) = 0flock(16, LOCK_SH) = 0flock(16, LOCK_UN) = 0flock(15, LOCK_UN) = 0flock(14, LOCK_UN) = 0flock(12, LOCK_UN) = 0flock(13, LOCK_UN) = 0
• for this example, compute phases are similar• no additional db.* files used
© 2007 Perforce Software 20
Execution Phase for Second Argument
open("depot/main/file1,v", O_RDONLY|O_LARGEFILE) = 17close(17) = 0open("depot/main/file2,v", O_RDONLY|O_LARGEFILE) = 17close(17) = 0open("depot/main/file3,v", O_RDONLY|O_LARGEFILE) = 17close(17) = 0flock(12, LOCK_EX) = 0flock(5, LOCK_EX) = 0flock(5, LOCK_UN) = 0flock(12, LOCK_UN) = 0
• second argument was //depot/branch1/...• lazy copies of //depot/main/...
• db.have (fd 12) updated for all six files syncedduring a single exclusive lock (journal is fd 5)
• batch updates introduced in 2003.2• extended to integrate in 2005.2
© 2007 Perforce Software 21
Command Cleanup Phase
close(4) = 0close(5) = 0close(9) = 0close(7) = 0close(6) = 0close(10) = 0close(11) = 0close(12) = 0close(13) = 0close(14) = 0close(15) = 0close(16) = 0close(8) = 0
• other system calls not traced, e.g. fsync()
© 2007 Perforce Software 22
Other Observations
• Locks taken and released as needed• not held over the entire command
• Type of db.* file lock varies throughout command• shared or exclusive• e.g. db.have
• Locks always taken in the same order• eliminates possibility of deadlock
• simpler and faster server code
© 2007 Perforce Software 23
Review?
• Claim: A command holding only shared locks will not block another from acquiring a shared lock
• Status: ?
© 2007 Perforce Software 24
Review?
• Claim: A command holding only shared locks will not block another from acquiring a shared lock
• Status: True
© 2007 Perforce Software 25
Review?
• Claim: A command holding only shared locks will not block another from acquiring a shared lock
• Status: True (directly) and false (indirectly)
© 2007 Perforce Software 26
"Wedge" Scenario
• Claim: A command holding only shared locks will not block another from acquiring a shared lock
• Status: True (directly) and false (indirectly)• A writer can "wedge" between readers
• writer acquires at least one exclusive lock• writer stalls acquiring an exclusive lock on a db.*
file for which a shared lock is held by a reader• other readers stall acquiring shared locks on db.*
files for which exclusive locks are held by the writer• Not a deadlock
© 2007 Perforce Software 27
Physical Memory
• Claim: If active server processes are swapped out, then additional physical memory is needed
• Status: ?
© 2007 Perforce Software 28
Physical Memory
• Claim: If active server processes are swapped out, then additional physical memory is needed
• Status: True
© 2007 Perforce Software 29
Physical Memory
• Claim: If active server processes are swapped out, then additional physical memory is needed
• Status: True• Claim: If active server processes are not
swapped out, then physical memory is adequate• Status: ?
© 2007 Perforce Software 30
Physical Memory
• Claim: If active server processes are swapped out, then additional physical memory is needed
• Status: True• Claim: If active server processes are not
swapped out, then physical memory is adequate• Status: Not necessarily true
© 2007 Perforce Software 31
Physical Memory
• Claim: If active server processes are swapped out, then additional physical memory is needed
• Status: True• Claim: If active server processes are not
swapped out, then physical memory is adequate• Status: Not necessarily true• For best performance, additional
physical memory required for filesystem cache
© 2007 Perforce Software 32
Filesystem Cache
• Satisfy I/O requests without physical I/O• logical I/O operations complete much faster, which• generally speeds various server phases, which• decreases time db.* file locks are held, which• leads to better concurrency, which• allows more commands processed faster
© 2007 Perforce Software 33
Filesystem Cache Content
• Consists of recently used pages from disk files• significant portion will be pages from db.* files
• Metadata accessed by traversing B-tree• only portions of the B-tree are read• whole db.* files are generally not cached
• Portions of db.* files not cached• unused client workspaces (portions of db.have)• retired release branches (portions of db.rev*)• etc.
© 2007 Perforce Software 34
Perforce and Filesystem Cache
• Server relies heavily on filesystem cache• no global cache within server
• less complex and more stable• similar functionality offered by filesystem cache
• Server does cache within each command• each db.* file opened has a private cache
• see "db.* File Generation Numbers" in2006 European User Conference
• additional caching within server data structures
© 2007 Perforce Software 35
Example of Filesystem Cache Benefit
• Benchmark using Solaris 10 server on Opteron• "forcedirectio" mount option simulates no cache
• Four concurrent "browse" clients on Linux 2.6• each "browse" issues dirs and fstat commands
from the top to the bottom of the repository• typical of drilling down using P4V
• each client does 64 "browses" without delay• random; same seeds used for each run• dedicated subnet
© 2007 Perforce Software 40
Filesystem Cache Effectiveness
• Goal is to minimize physical I/O for db.* files• high level of physical I/O might signal
the need for more physical memory• Monitor physical I/O with OS utilities
• iostat on Linux, FreeBSD, and Solaris• perfmon on Windows
© 2007 Perforce Software 41
Exclusionary Entries
• Claim: Exclusionary entries in protections and client views always improve performance
• Status: ?
© 2007 Perforce Software 42
Exclusionary Entries
• Claim: Exclusionary entries in protections and client views always improve performance
• Status: False
© 2007 Perforce Software 43
Exclusionary Entries
• Claim: Exclusionary entries in protections and client views always improve performance
• Status: False• Exclusionary entries: "-" starts depot path
• for example:list group dev * -//depot/product/main/doc/...
• Limiting files is recommended in thePerforce System Administrator's Guide• type of entry has performance implications
© 2007 Perforce Software 44
Exclusionary Entries
• Can significantly impact dirs command• more files scanned in the metadata
• General dirs behavior:• position using non-exclusionary entries• as directory is collected, skip to next directory
• Exclusionary entries necessitatescanning just to collect a directory• must scan to find a directory that's not excluded
• Wider exclusionary scope yields more scanning• subdirectories (recursively) included in scope
© 2007 Perforce Software 45
dirs Examplep4 dirs //depot/product/main/*
write group eng * //depot/product/main/...
//depot/product/main/art/file1.jpgfile2.jpgfile3.jpg
//depot/product/main/doc/file1.fmfile2.fmfile3.fm
//depot/product/main/src/file1.ccfile2.ccfile3.cc
© 2007 Perforce Software 46
dirs Examplep4 dirs //depot/product/main/*
write group eng * //depot/product/main/...
//depot/product/main/art/file1.jpgfile2.jpgfile3.jpg
//depot/product/main/doc/file1.fmfile2.fmfile3.fm
//depot/product/main/src/file1.ccfile2.ccfile3.cc
© 2007 Perforce Software 47
dirs Examplep4 dirs //depot/product/main/*
write group eng * //depot/product/main/...
//depot/product/main/art/file1.jpgfile2.jpgfile3.jpg
//depot/product/main/doc/file1.fmfile2.fmfile3.fm
//depot/product/main/src/file1.ccfile2.ccfile3.cc
© 2007 Perforce Software 48
dirs Examplep4 dirs //depot/product/main/*
write group eng * //depot/product/main/...
//depot/product/main/art//depot/product/main/art/file1.jpgfile2.jpgfile3.jpg
//depot/product/main/doc/file1.fmfile2.fmfile3.fm
//depot/product/main/src/file1.ccfile2.ccfile3.cc
© 2007 Perforce Software 49
dirs Examplep4 dirs //depot/product/main/*
write group eng * //depot/product/main/...
//depot/product/main/art/file1.jpgfile2.jpgfile3.jpg
//depot/product/main/doc/file1.fmfile2.fmfile3.fm
//depot/product/main/src/file1.ccfile2.ccfile3.cc
© 2007 Perforce Software 50
dirs Examplep4 dirs //depot/product/main/*
write group eng * //depot/product/main/...
//depot/product/main/art/file1.jpgfile2.jpgfile3.jpg
//depot/product/main/doc/file1.fmfile2.fmfile3.fm
//depot/product/main/src/file1.ccfile2.ccfile3.cc
© 2007 Perforce Software 51
dirs Examplep4 dirs //depot/product/main/*
write group eng * //depot/product/main/...
//depot/product/main/art/file1.jpgfile2.jpgfile3.jpg
//depot/product/main/doc/file1.fmfile2.fmfile3.fm
//depot/product/main/src/file1.ccfile2.ccfile3.cc
//depot/product/main/doc
© 2007 Perforce Software 52
dirs Examplep4 dirs //depot/product/main/*
write group eng * //depot/product/main/...
//depot/product/main/art/file1.jpgfile2.jpgfile3.jpg
//depot/product/main/doc/file1.fmfile2.fmfile3.fm
//depot/product/main/src/file1.ccfile2.ccfile3.cc
© 2007 Perforce Software 53
dirs Examplep4 dirs //depot/product/main/*
write group eng * //depot/product/main/...
//depot/product/main/art/file1.jpgfile2.jpgfile3.jpg
//depot/product/main/doc/file1.fmfile2.fmfile3.fm
//depot/product/main/src/file1.ccfile2.ccfile3.cc
© 2007 Perforce Software 54
dirs Examplep4 dirs //depot/product/main/*
write group eng * //depot/product/main/...
//depot/product/main/art/file1.jpgfile2.jpgfile3.jpg
//depot/product/main/doc/file1.fmfile2.fmfile3.fm
//depot/product/main/src/file1.ccfile2.ccfile3.cc
//depot/product/main/src
© 2007 Perforce Software 55
dirs Examplep4 dirs //depot/product/main/*
write group eng * //depot/product/main/...
//depot/product/main/art/file1.jpgfile2.jpgfile3.jpg
//depot/product/main/doc/file1.fmfile2.fmfile3.fm
//depot/product/main/src/file1.ccfile2.ccfile3.cc
© 2007 Perforce Software 56
dirs Examplep4 dirs //depot/product/main/*
write group eng * //depot/product/main/...
//depot/product/main/art/file1.jpgfile2.jpgfile3.jpg
//depot/product/main/doc/file1.fmfile2.fmfile3.fm
//depot/product/main/src/file1.ccfile2.ccfile3.cc
© 2007 Perforce Software 57
With Protection Exclusionsp4 dirs //depot/product/main/*
write group eng * //depot/product/main/...list group dev * -//depot/product/main/doc/...
//depot/product/main/art/file1.jpgfile2.jpgfile3.jpg
//depot/product/main/doc/file1.fmfile2.fmfile3.fm
//depot/product/main/src/file1.ccfile2.ccfile3.cc
© 2007 Perforce Software 58
With Protection Exclusionsp4 dirs //depot/product/main/*
write group eng * //depot/product/main/...list group dev * -//depot/product/main/doc/...
//depot/product/main/art/file1.jpgfile2.jpgfile3.jpg
//depot/product/main/doc/file1.fmfile2.fmfile3.fm
//depot/product/main/src/file1.ccfile2.ccfile3.cc
© 2007 Perforce Software 59
With Protection Exclusionsp4 dirs //depot/product/main/*
write group eng * //depot/product/main/...list group dev * -//depot/product/main/doc/...
//depot/product/main/art/file1.jpgfile2.jpgfile3.jpg
//depot/product/main/doc/file1.fmfile2.fmfile3.fm
//depot/product/main/src/file1.ccfile2.ccfile3.cc
© 2007 Perforce Software 60
With Protection Exclusionsp4 dirs //depot/product/main/*
write group eng * //depot/product/main/...list group dev * -//depot/product/main/doc/...
//depot/product/main/art//depot/product/main/art/file1.jpgfile2.jpgfile3.jpg
//depot/product/main/doc/file1.fmfile2.fmfile3.fm
//depot/product/main/src/file1.ccfile2.ccfile3.cc
© 2007 Perforce Software 61
With Protection Exclusionsp4 dirs //depot/product/main/*
write group eng * //depot/product/main/...list group dev * -//depot/product/main/doc/...
//depot/product/main/art/file1.jpgfile2.jpgfile3.jpg
//depot/product/main/doc/file1.fmfile2.fmfile3.fm
//depot/product/main/src/file1.ccfile2.ccfile3.cc
© 2007 Perforce Software 62
With Protection Exclusionsp4 dirs //depot/product/main/*
write group eng * //depot/product/main/...list group dev * -//depot/product/main/doc/...
//depot/product/main/art/file1.jpgfile2.jpgfile3.jpg
//depot/product/main/doc/file1.fmfile2.fmfile3.fm
//depot/product/main/src/file1.ccfile2.ccfile3.cc
© 2007 Perforce Software 63
With Protection Exclusionsp4 dirs //depot/product/main/*
write group eng * //depot/product/main/...list group dev * -//depot/product/main/doc/...
//depot/product/main/art/file1.jpgfile2.jpgfile3.jpg
//depot/product/main/doc/file1.fmfile2.fmfile3.fm
//depot/product/main/src/file1.ccfile2.ccfile3.cc
© 2007 Perforce Software 64
With Protection Exclusionsp4 dirs //depot/product/main/*
write group eng * //depot/product/main/...list group dev * -//depot/product/main/doc/...
//depot/product/main/art/file1.jpgfile2.jpgfile3.jpg
//depot/product/main/doc/file1.fmfile2.fmfile3.fm
//depot/product/main/src/file1.ccfile2.ccfile3.cc
© 2007 Perforce Software 65
With Protection Exclusionsp4 dirs //depot/product/main/*
write group eng * //depot/product/main/...list group dev * -//depot/product/main/doc/...
//depot/product/main/art/file1.jpgfile2.jpgfile3.jpg
//depot/product/main/doc/file1.fmfile2.fmfile3.fm
//depot/product/main/src/file1.ccfile2.ccfile3.cc
© 2007 Perforce Software 66
With Protection Exclusionsp4 dirs //depot/product/main/*
write group eng * //depot/product/main/...list group dev * -//depot/product/main/doc/...
//depot/product/main/art/file1.jpgfile2.jpgfile3.jpg
//depot/product/main/doc/file1.fmfile2.fmfile3.fm
//depot/product/main/src/file1.ccfile2.ccfile3.cc
//depot/product/main/src
© 2007 Perforce Software 67
With Protection Exclusionsp4 dirs //depot/product/main/*
write group eng * //depot/product/main/...list group dev * -//depot/product/main/doc/...
//depot/product/main/art/file1.jpgfile2.jpgfile3.jpg
//depot/product/main/doc/file1.fmfile2.fmfile3.fm
//depot/product/main/src/file1.ccfile2.ccfile3.cc
© 2007 Perforce Software 68
With Protection Exclusionsp4 dirs //depot/product/main/*
write group eng * //depot/product/main/...list group dev * -//depot/product/main/doc/...
//depot/product/main/art/file1.jpgfile2.jpgfile3.jpg
//depot/product/main/doc/file1.fmfile2.fmfile3.fm
//depot/product/main/src/file1.ccfile2.ccfile3.cc
© 2007 Perforce Software 69
With Protection and View Exclusionsp4 dirs -C //depot/product/main/*
//depot/product/main/... //dev-client/depot/product/main/...-//depot/product/main/art/... //dev-client/depot/product/main/art/...
write group eng * //depot/product/main/...list group dev * -//depot/product/main/doc/...
//depot/product/main/art/file1.jpgfile2.jpgfile3.jpg
//depot/product/main/doc/file1.fmfile2.fmfile3.fm
//depot/product/main/src/file1.ccfile2.ccfile3.cc
© 2007 Perforce Software 70
With Protection and View Exclusionsp4 dirs -C //depot/product/main/*
//depot/product/main/... //dev-client/depot/product/main/...-//depot/product/main/art/... //dev-client/depot/product/main/art/...
write group eng * //depot/product/main/...list group dev * -//depot/product/main/doc/...
//depot/product/main/art/file1.jpgfile2.jpgfile3.jpg
//depot/product/main/doc/file1.fmfile2.fmfile3.fm
//depot/product/main/src/file1.ccfile2.ccfile3.cc
© 2007 Perforce Software 71
With Protection and View Exclusionsp4 dirs -C //depot/product/main/*
//depot/product/main/... //dev-client/depot/product/main/...-//depot/product/main/art/... //dev-client/depot/product/main/art/...
write group eng * //depot/product/main/...list group dev * -//depot/product/main/doc/...
//depot/product/main/art/file1.jpgfile2.jpgfile3.jpg
//depot/product/main/doc/file1.fmfile2.fmfile3.fm
//depot/product/main/src/file1.ccfile2.ccfile3.cc
© 2007 Perforce Software 72
With Protection and View Exclusionsp4 dirs -C //depot/product/main/*
//depot/product/main/... //dev-client/depot/product/main/...-//depot/product/main/art/... //dev-client/depot/product/main/art/...
write group eng * //depot/product/main/...list group dev * -//depot/product/main/doc/...
//depot/product/main/art/file1.jpgfile2.jpgfile3.jpg
//depot/product/main/doc/file1.fmfile2.fmfile3.fm
//depot/product/main/src/file1.ccfile2.ccfile3.cc
© 2007 Perforce Software 73
With Protection and View Exclusionsp4 dirs -C //depot/product/main/*
//depot/product/main/... //dev-client/depot/product/main/...-//depot/product/main/art/... //dev-client/depot/product/main/art/...
write group eng * //depot/product/main/...list group dev * -//depot/product/main/doc/...
//depot/product/main/art/file1.jpgfile2.jpgfile3.jpg
//depot/product/main/doc/file1.fmfile2.fmfile3.fm
//depot/product/main/src/file1.ccfile2.ccfile3.cc
© 2007 Perforce Software 74
With Protection and View Exclusionsp4 dirs -C //depot/product/main/*
//depot/product/main/... //dev-client/depot/product/main/...-//depot/product/main/art/... //dev-client/depot/product/main/art/...
write group eng * //depot/product/main/...list group dev * -//depot/product/main/doc/...
//depot/product/main/art/file1.jpgfile2.jpgfile3.jpg
//depot/product/main/doc/file1.fmfile2.fmfile3.fm
//depot/product/main/src/file1.ccfile2.ccfile3.cc
© 2007 Perforce Software 75
With Protection and View Exclusionsp4 dirs -C //depot/product/main/*
//depot/product/main/... //dev-client/depot/product/main/...-//depot/product/main/art/... //dev-client/depot/product/main/art/...
write group eng * //depot/product/main/...list group dev * -//depot/product/main/doc/...
//depot/product/main/art/file1.jpgfile2.jpgfile3.jpg
//depot/product/main/doc/file1.fmfile2.fmfile3.fm
//depot/product/main/src/file1.ccfile2.ccfile3.cc
© 2007 Perforce Software 76
With Protection and View Exclusionsp4 dirs -C //depot/product/main/*
//depot/product/main/... //dev-client/depot/product/main/...-//depot/product/main/art/... //dev-client/depot/product/main/art/...
write group eng * //depot/product/main/...list group dev * -//depot/product/main/doc/...
//depot/product/main/art/file1.jpgfile2.jpgfile3.jpg
//depot/product/main/doc/file1.fmfile2.fmfile3.fm
//depot/product/main/src/file1.ccfile2.ccfile3.cc
© 2007 Perforce Software 77
With Protection and View Exclusionsp4 dirs -C //depot/product/main/*
//depot/product/main/... //dev-client/depot/product/main/...-//depot/product/main/art/... //dev-client/depot/product/main/art/...
write group eng * //depot/product/main/...list group dev * -//depot/product/main/doc/...
//depot/product/main/art/file1.jpgfile2.jpgfile3.jpg
//depot/product/main/doc/file1.fmfile2.fmfile3.fm
//depot/product/main/src/file1.ccfile2.ccfile3.cc
© 2007 Perforce Software 78
With Protection and View Exclusionsp4 dirs -C //depot/product/main/*
//depot/product/main/... //dev-client/depot/product/main/...-//depot/product/main/art/... //dev-client/depot/product/main/art/...
write group eng * //depot/product/main/...list group dev * -//depot/product/main/doc/...
//depot/product/main/art/file1.jpgfile2.jpgfile3.jpg
//depot/product/main/doc/file1.fmfile2.fmfile3.fm
//depot/product/main/src/file1.ccfile2.ccfile3.cc
//depot/product/main/src
© 2007 Perforce Software 79
With Protection and View Exclusionsp4 dirs -C //depot/product/main/*
//depot/product/main/... //dev-client/depot/product/main/...-//depot/product/main/art/... //dev-client/depot/product/main/art/...
write group eng * //depot/product/main/...list group dev * -//depot/product/main/doc/...
//depot/product/main/art/file1.jpgfile2.jpgfile3.jpg
//depot/product/main/doc/file1.fmfile2.fmfile3.fm
//depot/product/main/src/file1.ccfile2.ccfile3.cc
© 2007 Perforce Software 80
With Protection and View Exclusionsp4 dirs -C //depot/product/main/*
//depot/product/main/... //dev-client/depot/product/main/...-//depot/product/main/art/... //dev-client/depot/product/main/art/...
write group eng * //depot/product/main/...list group dev * -//depot/product/main/doc/...
//depot/product/main/art/file1.jpgfile2.jpgfile3.jpg
//depot/product/main/doc/file1.fmfile2.fmfile3.fm
//depot/product/main/src/file1.ccfile2.ccfile3.cc
© 2007 Perforce Software 81
Enumerated Inclusionary Protectionsp4 dirs //depot/product/main/*
write group dev * //depot/product/main/art/...write group dev * //depot/product/main/src/...write group doc * //depot/product/main/...write group qa * //depot/product/main/...write group rel * //depot/product/main/...
//depot/product/main/art/file1.jpgfile2.jpgfile3.jpg
//depot/product/main/doc/file1.fmfile2.fmfile3.fm
//depot/product/main/src/file1.ccfile2.ccfile3.cc
© 2007 Perforce Software 82
Enumerated Inclusionary Protectionsp4 dirs //depot/product/main/*
write group dev * //depot/product/main/art/...write group dev * //depot/product/main/src/...write group doc * //depot/product/main/...write group qa * //depot/product/main/...write group rel * //depot/product/main/...
//depot/product/main/art/file1.jpgfile2.jpgfile3.jpg
//depot/product/main/doc/file1.fmfile2.fmfile3.fm
//depot/product/main/src/file1.ccfile2.ccfile3.cc
© 2007 Perforce Software 83
Enumerated Inclusionary Protectionsp4 dirs //depot/product/main/*
write group dev * //depot/product/main/art/...write group dev * //depot/product/main/src/...write group doc * //depot/product/main/...write group qa * //depot/product/main/...write group rel * //depot/product/main/...
//depot/product/main/art/file1.jpgfile2.jpgfile3.jpg
//depot/product/main/doc/file1.fmfile2.fmfile3.fm
//depot/product/main/src/file1.ccfile2.ccfile3.cc
© 2007 Perforce Software 84
Enumerated Inclusionary Protectionsp4 dirs //depot/product/main/*
write group dev * //depot/product/main/art/...write group dev * //depot/product/main/src/...write group doc * //depot/product/main/...write group qa * //depot/product/main/...write group rel * //depot/product/main/...
//depot/product/main/art/file1.jpgfile2.jpgfile3.jpg
//depot/product/main/doc/file1.fmfile2.fmfile3.fm
//depot/product/main/src/file1.ccfile2.ccfile3.cc
//depot/product/main/art
© 2007 Perforce Software 85
Enumerated Inclusionary Protectionsp4 dirs //depot/product/main/*
write group dev * //depot/product/main/art/...write group dev * //depot/product/main/src/...write group doc * //depot/product/main/...write group qa * //depot/product/main/...write group rel * //depot/product/main/...
//depot/product/main/art/file1.jpgfile2.jpgfile3.jpg
//depot/product/main/doc/file1.fmfile2.fmfile3.fm
//depot/product/main/src/file1.ccfile2.ccfile3.cc
© 2007 Perforce Software 86
Enumerated Inclusionary Protectionsp4 dirs //depot/product/main/*
write group dev * //depot/product/main/art/...write group dev * //depot/product/main/src/...write group doc * //depot/product/main/...write group qa * //depot/product/main/...write group rel * //depot/product/main/...
//depot/product/main/art/file1.jpgfile2.jpgfile3.jpg
//depot/product/main/doc/file1.fmfile2.fmfile3.fm
//depot/product/main/src/file1.ccfile2.ccfile3.cc
© 2007 Perforce Software 87
Enumerated Inclusionary Protectionsp4 dirs //depot/product/main/*
write group dev * //depot/product/main/art/...write group dev * //depot/product/main/src/...write group doc * //depot/product/main/...write group qa * //depot/product/main/...write group rel * //depot/product/main/...
//depot/product/main/art/file1.jpgfile2.jpgfile3.jpg
//depot/product/main/doc/file1.fmfile2.fmfile3.fm
//depot/product/main/src/file1.ccfile2.ccfile3.cc
//depot/product/main/src
© 2007 Perforce Software 88
Enumerated Inclusionary Protectionsp4 dirs //depot/product/main/*
write group dev * //depot/product/main/art/...write group dev * //depot/product/main/src/...write group doc * //depot/product/main/...write group qa * //depot/product/main/...write group rel * //depot/product/main/...
//depot/product/main/art/file1.jpgfile2.jpgfile3.jpg
//depot/product/main/doc/file1.fmfile2.fmfile3.fm
//depot/product/main/src/file1.ccfile2.ccfile3.cc
© 2007 Perforce Software 89
Enumerated Inclusionary Protectionsp4 dirs //depot/product/main/*
write group dev * //depot/product/main/art/...write group dev * //depot/product/main/src/...write group doc * //depot/product/main/...write group qa * //depot/product/main/...write group rel * //depot/product/main/...
//depot/product/main/art/file1.jpgfile2.jpgfile3.jpg
//depot/product/main/doc/file1.fmfile2.fmfile3.fm
//depot/product/main/src/file1.ccfile2.ccfile3.cc
© 2007 Perforce Software 90
Enumerated Inclusionary Protections and Viewsp4 dirs -C //depot/product/main/*
//depot/product/main/doc/... //dev-client/depot/product/main/doc/...//depot/product/main/src/... //dev-client/depot/product/main/src/...
write group dev * //depot/product/main/art/...write group dev * //depot/product/main/src/...write group doc * //depot/product/main/...write group qa * //depot/product/main/...write group rel * //depot/product/main/...
//depot/product/main/art/file1.jpgfile2.jpgfile3.jpg
//depot/product/main/doc/file1.fmfile2.fmfile3.fm
//depot/product/main/src/file1.ccfile2.ccfile3.cc
© 2007 Perforce Software 91
Enumerated Inclusionary Protections and Viewsp4 dirs -C //depot/product/main/*
//depot/product/main/doc/... //dev-client/depot/product/main/doc/...//depot/product/main/src/... //dev-client/depot/product/main/src/...
write group dev * //depot/product/main/art/...write group dev * //depot/product/main/src/...write group doc * //depot/product/main/...write group qa * //depot/product/main/...write group rel * //depot/product/main/...
//depot/product/main/art/file1.jpgfile2.jpgfile3.jpg
//depot/product/main/doc/file1.fmfile2.fmfile3.fm
//depot/product/main/src/file1.ccfile2.ccfile3.cc
© 2007 Perforce Software 92
Enumerated Inclusionary Protections and Viewsp4 dirs -C //depot/product/main/*
//depot/product/main/doc/... //dev-client/depot/product/main/doc/...//depot/product/main/src/... //dev-client/depot/product/main/src/...
write group dev * //depot/product/main/art/...write group dev * //depot/product/main/src/...write group doc * //depot/product/main/...write group qa * //depot/product/main/...write group rel * //depot/product/main/...
//depot/product/main/art/file1.jpgfile2.jpgfile3.jpg
//depot/product/main/doc/file1.fmfile2.fmfile3.fm
//depot/product/main/src/file1.ccfile2.ccfile3.cc
© 2007 Perforce Software 93
Enumerated Inclusionary Protections and Viewsp4 dirs -C //depot/product/main/*
//depot/product/main/doc/... //dev-client/depot/product/main/doc/...//depot/product/main/src/... //dev-client/depot/product/main/src/...
write group dev * //depot/product/main/art/...write group dev * //depot/product/main/src/...write group doc * //depot/product/main/...write group qa * //depot/product/main/...write group rel * //depot/product/main/...
//depot/product/main/art/file1.jpgfile2.jpgfile3.jpg
//depot/product/main/doc/file1.fmfile2.fmfile3.fm
//depot/product/main/src/file1.ccfile2.ccfile3.cc
//depot/product/main/src
© 2007 Perforce Software 94
Enumerated Inclusionary Protections and Viewsp4 dirs -C //depot/product/main/*
//depot/product/main/doc/... //dev-client/depot/product/main/doc/...//depot/product/main/src/... //dev-client/depot/product/main/src/...
write group dev * //depot/product/main/art/...write group dev * //depot/product/main/src/...write group doc * //depot/product/main/...write group qa * //depot/product/main/...write group rel * //depot/product/main/...
//depot/product/main/art/file1.jpgfile2.jpgfile3.jpg
//depot/product/main/doc/file1.fmfile2.fmfile3.fm
//depot/product/main/src/file1.ccfile2.ccfile3.cc
© 2007 Perforce Software 95
Enumerated Inclusionary Protections and Viewsp4 dirs -C //depot/product/main/*
//depot/product/main/doc/... //dev-client/depot/product/main/doc/...//depot/product/main/src/... //dev-client/depot/product/main/src/...
write group dev * //depot/product/main/art/...write group dev * //depot/product/main/src/...write group doc * //depot/product/main/...write group qa * //depot/product/main/...write group rel * //depot/product/main/...
//depot/product/main/art/file1.jpgfile2.jpgfile3.jpg
//depot/product/main/doc/file1.fmfile2.fmfile3.fm
//depot/product/main/src/file1.ccfile2.ccfile3.cc
© 2007 Perforce Software 96
Embedded Wildcards
• Claim: For best performance, protections and client views must be concise
• Status: ?
© 2007 Perforce Software 97
Embedded Wildcards
• Claim: For best performance, protections and client views must be concise
• Status: False
© 2007 Perforce Software 98
Embedded Wildcards
• Claim: For best performance, protections and client views must be concise
• Status: False• Example usage:
write group doc * //depot/.../doc/...
• Results are typically more concise• might be easier to understand
• Can require additional CPU and I/O• when protections table joined with client view
© 2007 Perforce Software 99
Minimize Embedded Wildcards
• Repository structure not consistent with usage• examples:
• documentation scattered within source• platform-specific scattered within generic
• Restructuring might be better solution, or ...
© 2007 Perforce Software 100
Enumerating Entries
• Replace with explicitly enumerated entries• provides same (dataset-specific) functionality• allows for "fine-tuning"
• Typically increases size• increased protection table size tolerated well• increased client view size perhaps not so well
• Performance improvement can be significant• following example for sync compute phase
• 7000 files returned
© 2007 Perforce Software 111
Example Protection Table Entries
• Using embedded wildcards:write group dev * //depot/main/.../00/...write group dev * //depot/main/.../01/......write group dev * //depot/main/.../08/...write group dev * //depot/main/.../09/...
• Using enumerated entries:write group dev * //depot/main/00/00/...write group dev * //depot/main/00/01/......write group dev * //depot/main/00/08/...write group dev * //depot/main/00/09/...write group dev * //depot/main/01/00/...write group dev * //depot/main/01/01/......write group dev * //depot/main/99/08/...write group dev * //depot/main/99/09/...
© 2007 Perforce Software 112
Example Client View Entries
• Using embedded wildcards://depot/main/.../00/... //dev-client/depot/main/.../00/...
• Using enumerated entries://depot/main/00/00/... //dev-client/depot/main/00/00/...//depot/main/01/00/... //dev-client/depot/main/01/00/......//depot/main/98/00/... //dev-client/depot/main/98/00/...//depot/main/99/00/... //dev-client/depot/main/99/00/...
© 2007 Perforce Software 113
Summary
• Claim: Perforce Server uses metadatastored in the db.* files
• Status: True
• Claim: There is a single global lock controllingall metadata access
• Status: False
© 2007 Perforce Software 114
Summary
• Claim: The single global lock is an exclusive lock,serializing all metadata access
• Status: False
• Claim: A command holding only shared locks will not block another from acquiring a shared lock
• Status: True (directly) and false (indirectly)
© 2007 Perforce Software 115
Summary
• Claim: If active server processes are swapped out, then additional physical memory is needed
• Status: True
• Claim: If active server processes are not swapped out, then physical memory is adequate
• Status: Not necessarily true
© 2007 Perforce Software 116
Summary
• Claim: Exclusionary entries in protections and client views always improves performance
• Status: False
• Claim: For best performance, protections and client views must be concise
• Status: False