How to reduce downtime for consistency check and fix CQ5.4 / WEM

Use case: Consistency check and fix takes days to complete some time, We can not afford this much down time.

Approach:  There are different kind of inconsistencies like,

Orphaned child

Error message : NodeState CHILD_UUID references inexistent parent uuid PARENT_UUID

Current solution : restore entire backup

Parent referencing inexistent child

Error message : NodeState PARENT_UUID references inexistent child {}CHILD_NAME with id CHILD_UUID

Current solution : consistency check/fix should repair such issue Or if you know UUID then slective UUID check

and fix should resolve this issue.

Child referencing invalid parent

Error message : ChildNode has invalid parent uuid: INVALID_PARENT_UUID (instead of VALID_PARENT_UUID)

Current solution : Use CRX Console

Parent not referencing existing child

Error message : javax.jcr.ItemNotFoundException: failed to build path of CHILD_UUID: PARENT_UUID has no child

entry for CHILD_UUID

Current solution : Use CRX Console

Node already exists

Error message : javax.jcr.ItemExistsException: at

org.apache.jackrabbit.core.NodeImpl.internalAddChildNode(NodeImpl.java:766)Current solution : consistency check/fix should repair such issueSearch index inconsistencyError message : WARN NodeIteratorImpl: Exception retrieving Node with UUID:
003171fe-e2e8-457b-a3af-f74eed12c1b9: javax.jcr.ItemNotFoundException:
003171fe-e2e8-457b-a3af-f74eed12c1b9Current solution : recreate search index or use redo.log

More information can be found here, http://dev.day.com/content/kb/home/Crx/Troubleshooting/RepositoryInconsistency.html
Problem with full consistency check and fix is it even traverse nodes which are not corrupted. And key to do fast consistency check and fix is finding all problematic entry in advance and then doing selective consistency check and fix. Another problem is corruption will not be revealed unless node is accessed.
CRX already provide way to do consistency check and fix using UI, But it does not resolve all inconsistency, though you can use this feature to touch each node and find inconsistencies 

For CQ5.4 or CQ5.5

Or use following CURL command

Phase 1: Finding all entry.

rm login.txt

curl -c login.txt "http://<host>:<port>/crx/login.jsp?UserId=admin&Password=admin"

rm progress.txt

curl -b login.txt -f -o progress.txt -d "type=traversal&traversalLogEach=true&fixInconsistencies=true&rootNode=/" "http://<host>:<port>/crx/config/check_frame.jsp"

Phase 2: Then You can use following program to find out all inconsistencies in log

• You can use attached file (Unzip files) to monitor your crx log (/logs/crx/error.log in CQ5.4 and /logs/error.log in CQ5.5) file and send an email in case something is wrong. To use this application

• Compile java file using javac -classpath .:mail.jar:activation.jar FindCorruption.java

• Create a cron Job that runs every hour to monitor your log file (For windows you can use service)

• Use following command to run command java -classpath .:mail.jar:activation.jar FindCorruption

• Make sure that You put all the files attached in same location Or change PROPERTY_FILE_LOCATION property in FindCorruption.java

• Configure your mail server and output folder location information in findCurruption.properties. For configuring email server please check http://dev.day.com/docs/en/cq/current/administering/notification.html

• What you expect is an email when program find corruption pattern in log. This also send you location of all files you need to quickly fix inconsistencies "that can be fixed by consistency check and fix"

You can modify this program to just scan logs and output instruction some where else.

Simple Java Program to find pattern can be found here

Note: This is just an example, You can write your own version using powerful parsing scripting languages to do same task. Again this is not supported by Adobe or by Me, Feel free to add your comment though of how it can be improved.