How to improve Tar Optimization and Indexing process in CQ 5.4 / WEM

Use Case: You have a lot of tar files (Because of some bad code) that you want to clear quickly Or you want to improve indexing process while reindexing repository.

Pre-requisite CRX2.2 with Hotfix 2.2.0.46 or greater.

Solution

There are two new TarPersistenceManager configuration
options.

The optimizeCount option controls how many blocks the
TarPM optimizer processes at a time. The default value is
one, which avoids blocking concurrent user operations as
much as possible. By increasing this value the TarPM
optimization speed increases, but the performance of
concurrent user operations decreases.

In addition to this you can do following,

The main problem of the Tar PM optimization is reading from the Tar index files in random order. To speed this up, load the files in the buffer cache every few minutes, from both the crx.default and the version directory:
cat .../repository/workspaces/crx.default/index*.tar > /dev/null
cat .../repository/version/index*.tar > /dev/null


The indexInMemory option is a boolean flag for making the
repository read all TarPM index files entirely in memory.
This can significantly boost performance for large
repositories, but requires gigabytes of free RAM.

Here is How this needs to be set,

<PersistenceManager class="com.day.crx.persistence.tar.TarPersistenceManager">
<param name="optimizeCount" value="{Numeric value you want to set}" />
<param name="indexInMemory" value="true" />
</TarPersistenceManager>


Please test it before use

Special Thanks to Thomas Mueller and Jukka zitting from Adobe for this information.

How to use static agent in CQ / WEM

Use case:
1) You want to create file system view of repository on each activation.
2) You want to create backup of your site in file system (After that you can zip it and store it where ever you want)

Solution:

1) Login in to your system and then go to http://host:port/libs/wcm/core/content/misc.html#/etc/replication/agents.author
2) Create a new replication agent of type "Static Delivery Agent"



3) Double click on newly created agent and click on edit. Enter all description and click on "Rules" tab.
4) You can enter target repository (By default it is /tmp in unix system). Make sure that you have sufficient rights to that folder.
5) In definition section, specify for a specific path what rule you want to apply. For example if you have rule like
/content/geo* ${path}.html?wcmmode=preview this mean for any {path} activated that start with /content/geo* static agent will request http://host:port/{path}.html?wcmmode=preview and then store that content under {target path} you set above




6) Now we have to override default static setting from felix console. For that please go to http://host:port/system/console/configMgr and click on "Static Content Builder". Default port there is 4502, You can change that according to your need.



7) Now after each activation static representation of content will get stored in {target path}

8) Note that static agent get trigger on replicate action. So if you configure static agent on publish and activate page on author, It will not get triggered. In order to trigger this on publish, Please create static agent under agent.publish (http://HOST:PORT/miscadmin#/etc/replication/agents.publish) and then add "triggerModified" property to true using CRXDE light. You can also remove ?wcmmode=preview from publish, From set of rules (Do not forget to change felix configuration to point to publish instance)




Note that modification event only works for cq:page. For all other modification to get trigger on publish, Please refer http://dev.day.com/content/kb/home/cq5/CQ5SystemAdministration/HowToFlushAssetsPublish.html

You might have to entry of agent:static in this case.

Note that if you are making above changes in autor, You have to activate static agent on publish after making change.


You can write a code to iterate through all pages in repository and then use replication API to activate those pages. This way you can create file system snapshot of whole repository. You can use following code to call replication through replication,

com.day.cq.replication.Replicator replicator = sling.getService(Replicator.class) // in a JSP

OR

@Reference
com.day.cq.replication.Replicator replicator = null; // in an OSGi component

THEN

com.day.cq.replication.ReplicationOptions opts = new ReplicationOptions();
opts.setFilter(new com.day.cq.replication.AgentFilter() {
public boolean isIncluded(com.day.cq.replication.Agent agent) {
return agent.getId("static"); // the ID of the agent is the node name, e.g. "static" for /etc/replication/agents/static
}
}
); // filter by replication agent

THEN

replicator.replicate(javax.jcr.Session, ReplicationActionType.ACTIVATE, path, opts); // activate

Or

use following code to get all static replication agent

AgentManager agentMgr = sling.getService(AgentManager.class);
ArrayList allStaticAgent = new ArrayList();
for (Agent agent: agentMgr.getAgents().values()) {
if (agent.isEnabled()) {
if(agent.getId().equalsIgnoreCase("static"){
allStaticAgent.add(agent);
}
}

}
}

THEN

ReplicationOptions replicationOptions = new ReplicationOptions();
for(Agent agent:allStaticAgent){
replicationOptions.setFilter(new AgentIdFilter(agent.getId()));
replicator.replicate(javax.jcr.Session, ReplicationActionType.ACTIVATE, path, opts); // activate

}

Note For creating file system view of DAM renditions or DAM Asset you might have to create some custom plugin.

How to remove Geometrixx from CQ5.4 / WEM

Use Case You want to remove default geometrixx site comes with OOTB CQ

Solution

1) Download following package



2) Install it using package manager

In CQ5.5

In CQ5.5 this package is provided OOTB. Please look for

cq-geometrixx-pkg-5.5.4.zip and 

cq-geometrixx-outdoors-pkg-5.5.8.zip package 

In CQ 5.6

From package manager uninstall cq-geometrixx-all-pkg-5.6.6.zip

You can uninstall them to remove geometrixx. 

In 5.6 you can also start CQ with following command line to start CQ without geometrixx 
java -jar <jar file name> -r publish,nosamplecontent

Important Note: Please test it in local before using it in production like environment

Special Thanks to Micheal Marth from Adobe for this information.

How to stop CQ to retry replication queue in CQ / WEM

Use case:

Case 1: You are doing an upgrade project of publish instance. And you have created some dummy publish agent that will take replication request and keep it in queue. You forgot to change retry limit while creating the replication agent and now every 5 minute ( Or default retry time interval) replication event is trying to clear to queue. Now because of this all other valid replication agent is going through slowly.

Case 2: One or more publish instance is down and replication queue is queuing up all the request. Again it is trying to retry replication job after default time period which slows down other replication queue.

Assumption : You are using CQ5.4 or CQ5.3 with replication fix pack.

How it works : CQ uses Ordered processing of replication jobs



Also each job has retry limit associated with it



So CQ uses top most job retry value to retry. So in order to set retry value of queue you need to change retry value of top most Job.

Solution :
For case 1, If you are trying to create new dummy publish agent, Make sure that before enabling it you set the retry value in replication agent



If in case you forget to do that you can use below package to do that.


Case 2: For this you can use attached package to set or reset retry limit.
Download attached package using package manager and then go to http://<host>:<port>/apps/tools/retryset/run.html (you can use dry run to check first)



This is how UI will look like after run




Please test it before use