How to Use Crypto support in Adobe CQ / AEM

Use Case: You want to protect sensitive information in OSGI configuration

Solution: CQ > 5.5 (Granite platform) introduces a new crypto cupport service (com.adobe.granite.crypto.CryptoSupport) to protect sensitive information.

To store protected configuration, the Apache Felix Web Console should be used.



to unprotected data you can use CryptoSupport.unprotect(String) method.

Example


@Component
public class Test {
@Reference
private CryptoSupport cryptoSupport;
@Activate
@Modified
private void configure(Map config) {
final String protectedConfig = config.get("password");
final String plainTextConfig;
if (this.cryptoSupport.isProtected(protectedConfig)) {
plainTextConfig = this.cryptoSupport.unprotect(protectedConfig);
} else {
plainTextConfig = protectedConfig;
}
}
}


You can also use crypto support JSON call to get data. For example following curl command will return protected sting you can use

$ curl -uadmin:admin -F datum=password http://localhost:4502/system/console/crypto/.json
{"protected": "{4dd7095d321134b5e6737311fa82afaa335390762e43136ee8acb3897296865d}"}

Note: Crypt generated on one machine will not work on other machine as each one has different Key. In order to make key work across all instance, You can create package of /etc/key and install it in all instances and then restart "com.adobe.granite.crypto" bundle from system console.

If you want to deploy these key as part of code across all instances then first down load hmac and master binary from /etc/key

then create a node under /etc/key in your file system (Code repo)

<?xml version="1.0" encoding="UTF-8"?>
<jcr:root xmlns:sling="http://sling.apache.org/jcr/sling/1.0" xmlns:jcr="http://www.jcp.org/jcr/1.0" xmlns:rep="internal"
 jcr:mixinTypes="[rep:AccessControllable]"
 hmac="{Binary}"
 hidden="{Boolean}true"
 master="{Binary}"
 jcr:primaryType="sling:Folder"/>

under /etc/key add two files name "hmac.binary" and "master.binary" that you copied from system where secret was generated.

Deploy your code. Make sure to restart "com.adobe.granite.crypto" for very first time you upload these key. (You can also do this using CURL command)

Crypto Suport API: http://dev.day.com/docs/en/cq/current/javadoc/com/adobe/granite/crypto/package-summary.html

How to create encoded password that you can use for curl in CQ / WEM

Use Case: In Curl command most of the time you have to enter user credentials to perform CRUD operation. Some people might have security concern sharing admin password with other users.

Solution: You can use following java program to create encrypted password that you can use in curl command

import java.io.UnsupportedEncodingException;


public class Test {
public static void main(String[] args) {
String password = "admin";
try {
for (byte x : password.getBytes("UTF-8")) {
System.out.print("%" + Integer.toHexString((x & 255) + 256).substring(1));
}
} catch (UnsupportedEncodingException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}

}
}

1) Create Test.java
2) Copy paste above code
3) Run program by changing password value (Or pass it through command line arg)
4) Use output of program to use it in CURL

Special thanks to Thomas Mueller From Adobe for providing this information.

How to do user management using POST API / CURL in CQ5.5

Use case There is a new POST API you can use to do user / Group / ACL / Profile management in CQ5.5

Solution You can use POST command (Using Curl to perform these operations as well)

HttpClient client = new HttpClient();
client.getState().setCredentials(
new AuthScope(AuthScope.ANY_HOST, AuthScope.ANY_PORT),
new UsernamePasswordCredentials("admin", "admin"));


To create User
PostMethod post = new PostMethod("http://localhost:4502/libs/granite/security/post/authorizables");
post.addParameter("createUser", "");
post.addParameter("authorizableId", "<uid>");
post.addParameter("rep:password", "<pwd>");

You can do following set of operations using /libs/granite/security/post/authorizables


deleteAuthorizable
createUser
createGroup

Add member
PostMethod post = new PostMethod("http://localhost:4502/home/groups/m/mygroup.rw.html");
post.addParameter("addMembers", user);

Remove Member
PostMethod post = new PostMethod("http://localhost:4502/home/groups/m/mygroup.rw.html");
post.addParameter("removeMembers", user);

You can do following set of operations using /home/<Path to user or group>.rw.html

disableUser
addImpersonators
removeImpersonators
addMembers
removeMembers
membership

This is just few thing you can do using new Security API. There are many services to perform read operations. Complete set is available with Very Nice detailed Java Doc under,

com.adobe.granite.security.user.servlets

Links to come.

Some Curl Command example

Get User Info

$ curl -u admin:admin http://localhost:4502/libs/granite/security/<username>.json

Create User

$ curl -u admin:admin -FcreateUser= -FauthorizableId=testuser -Frep:password=test http://localhost:4502/libs/granite/security/post/authorizables

Create Group

$ curl -u admin:admin -FcreateGroup= -FauthorizableId=testGroup1 http://localhost:4502/libs/granite/security/post/authorizables

Create user with Profile

$ curl -u admin:acreateUser= -FauthorizableId=testuser -Frep:password=test -Fprofile/<Property>=propertyvalue http://localhost:4502/libs/granite/security/post/authorizables

Set a Profile Property on an Existing User

$ curl -u admin:admin -Fprofile/<Property>=propertyvalue http://localhost:4502/home/users/t/testuser1.rw.html

Create a User as a Member of a Group

$ curl -u admin:admin -FcreateUser= -FauthorizableId=testuser -Frep:password=mypassword -Fmembership=contributor http://localhost:4502/libs/granite/security/post/authorizables

Add a User to a Group

$ curl -u admin:admin -FaddMembers=testuser http://localhost:4502/home/groups/t/testGroup.rw.html

Remove a User from a Group

$ curl -u admin:admin -FremoveMembers=testuser http://localhost:4502/home/groups/t/testGroup.rw.html

Set a User’s Group Memberships

$ curl -u admin:admin -Fmembership=contributor -Fmembership=author http://localhost:4502/home/users/t/testuser.rw.html

Delete user and Group

$ curl -u admin:admin -FdeleteAuthorizable= http://localhost:4502/home/users/t/testuser

$ curl -u admin:admin -FdeleteAuthorizable= http://localhost:4502/home/groups/t/testGroup

Permission Management (CQ5.6 onward):

http://sling.apache.org/documentation/bundles/managing-permissions-jackrabbit-accessmanager.html

For previous version you can try to use jar file from http://mvnrepository.com/artifact/org.apache.sling/org.apache.sling.jcr.jackrabbit.accessmanager

Read Permission:

$ curl -u admin:admin -F:applyTo=myuser http://localhost:4502/<Path>.acl.json

OR

$ curl -u admin:admin -F:applyTo=myuser http://localhost:4502/<Path>.eacl.json


Delete Permission:

$ curl -u admin:admin -F:applyTo=myuser http://localhost:4502/<Path>.deleteAce.html

Modify Permission:

$ curl -u admin:admin -FprincipalId=<Some User> -Fprivilege@jcr:all=granted http://localhost:4502/<Path>.modifyAce.html

API Doc:
http://dev.day.com/docs/en/cq/5-5/javadoc/com/adobe/granite/security/user/servlets/AuthorizableServlet.html

Sling Doc:

http://sling.apache.org/site/managing-users-and-groups-jackrabbitusermanager.html

Code:

http://svn.apache.org/viewvc/sling/trunk/bundles/jcr/jackrabbit-accessmanager/src/main/java/org/apache/sling/jcr/jackrabbit/accessmanager/post/

Special Thanks to Justin Edelson from Adobe to provide this information

How to run online backup / Datastore GC / Tar Optimization using curl in CQ5.5

Use case: Curl command for online backup / Datastore GC / Tar Optimization for CQ5.4 does not work for CQ5.5 any more

Reason: Many CRX related operation is moved to JMX. Now you can use any JMX console or JMX API to invoke these operations.

Solution: You can use following curl commands


For online backup


curl -u <UID>:<PASSWORD> -X POST http://<HOST>:<PORT>/system/console/jmx/com.adobe.granite:type=Repository/op/startBackup/java.lang.String?target=<PATH OF BACKUP>/test.zip

Following options are available for backup

Delay (Default is 10 mili second)

curl -u <UID>:<PASSWORD> -X POST
http://<HOST>:<PORT>/system/console/jmx/com.adobe.granite:type=Repository/a/BackupDelay?value=<TIME-IN-MILISECOND>


Note After running this from front end you will see (null) as response. You have to check CRX log to verify if backup started. In CRX log (/crx-quickstart/logs/error.log) you would see message like this

*INFO* [Backup Worker Thread] com.day.crx.core.backup.Backup Backup started.
*INFO* [Backup Worker Thread] com.day.crx.core.backup.Backup Read size (1.234 gb) in 1.97s


OR Use following CURL for online backup


curl -u admin:admin --data "delay=3&force=false&target=test3.zip" http://HOST:PORT/libs/granite/backup/content/admin/backups/

Where,
delay: Time delay between write block.
force: To override last one
Target: Absolute path of backup. If path is not provided and just file name is given, Then it will use current path of installation.

Once backup is started, You can cancel it using following curl command

curl -u admin:admin http://HOST:PORT/libs/granite/backup/content/admin/backups.cancel.html

You can also track online progress using following URL

http://HOST:PORT/libs/granite/backup/content/admin.html




For Datastore GC


curl -u <UID>:<PASSWORD> -X POST http://<HOST>:<PORT>/system/console/jmx/com.adobe.granite:type=Repository/op/runDataStoreGarbageCollection/java.lang.Boolean

To delete data and delay as 2

curl -u admin:admin -X POST --data "delete=true&delay=2" http://HOST:PORT/system/console/jmx/com.adobe.granite%3Atype%3DRepository/op/runDataStoreGarbageCollection/java.lang.Boolean

CRX Log message to look for

*INFO* [127.0.0.1 [1332343886706] POST /system/console/jmx/com.adobe.granite%3Atype%3DRepository/op/runDataStoreGarbageCollection/java.lang.Boolean HTTP/1.1] com.day.crx.sling.server.impl.jmx.GarbageCollection Scanning /libs/wcm/core/content/siteadmin/actions/create/menu/createPage

Automate DataStore GC (AEM 5.6 onward)

Option 1: Using Configuration

create new configuration name 

<some path for config>/com.day.crx.sling.server.impl.jmx.GarbageCollectionConfig-DatastoreGarbageCollector.xml with following values

<?xml version="1.0" encoding="UTF-8"?>
<jcr:root xmlns:sling="http://sling.apache.org/jcr/sling/1.0" xmlns:cq="http://www.day.com/jcr/cq/1.0" xmlns:jcr="http://www.jcp.org/jcr/1.0"
    jcr:primaryType="sling:OsgiConfig"
    jobName="Datastore Garbage Collector"
    schedule="00 00 23 ? * SUN"
    deleteNodes="{Boolean}true" />

This mean that run Datastore Garbage collection every Sunday at 11 PM.

Option 2: Using CURL and CRON job. Create a Cron job on your machine and run CURL command given above.

 
For Tar optimization

curl -u <UID>:<PASSWORD> -X POST http://<HOST>:<PORT>/system/console/jmx/com.adobe.granite:type=Repository/op/startTarOptimization/

CRX Log Message to look for

com.day.crx.persistence.tar.file.TarFile /crx-quickstart/repository/workspaces/crx.default/data_00000.tar id:0 length:75436032 append:75435008 943288331 optimize: 2420224


Similarly you can use CURL command to different JMX operation listed in

http://<HOST>:<PORT>/system/console/jmx/com.adobe.granite:type=Repository






How to identify master using Curl Command And then run backup only on master

ISMASTER=`curl -s -u admin:admin http://HOST:PORT/system/console/jmx/com.adobe.granite:type=Repository | grep -A 1 crx.cluster.master | sed 's/\<\/*td\>//g' | awk 'NR>1' | awk '{ print $1 }'`
if [ $ISMASTER != 'true' ]; then
echo "Not running backup, that is the slave node";
exit 1;
else
echo "Running online backup this is master Node"
# Your curl command for online backup goes here
fi;

Special Thanks to Andrew Khoury from Adobe for providing this CURL command.


Use firebug to get complete URL and then replace it in above curl command.

How To Integrate a SOAP Web-Service Toolkit with Adobe CQ / WEM

Very nice blog post for this can be found here

For CXF

http://shsteimer.com/blog/2012/03/integrating-a-soap-web-service-toolkit-with-day-cq/

http://cqblog.inside-solutions.ch/2013/09/04/consuming-webservices-using-apache-cxf-in-cq5-6-1/

For JAX-WS

http://cqblog.inside-solutions.ch/2013/11/01/consuming-soap-web-service-with-jax-ws-in-cq5-6-1/

For Axis 2

Method 1
1) Create Axis2 Fragment bundle using this pom.xml (You might have to change Export-Package section based on your project)
2) Already created Axis2 fragment bundle can be downloaded from here (This is working for OOTB CQ5.4)
3) Put this to install folder of your application. Make sure that it is in fragment state in OSGI
4) Restart your instance (Fragment bundle resolution will take place upon restart)



Note If you have custom jar that uses Axis2. You can put them under libs folder (To same level where your pom.xml or bnd file is).

Make sure that you have following entry in bnd file

Import-Package: *;resolution:=optional
Private-Package: *
Dynamic-Import: *
Embed-Dependency: *;scope=compile|runtime
Embed-Directory: /libs
Embed-Transitive: true

Method 2 (Method 1 is recommended)
If you are keen to use Axis2 with CQ5.X. I have compiled set of OSGI bundle that you can use for this.

1) Download all bundles from here
2) Put them in /apps//install folder
3) Use Axis2 API in your project
4) All Jar file (Non OSGI) can be found here
5) Also see how to convert Jar file in to OSGI bundle
6) Make sure that you put all non OSGI class file in bundle class path of your custom OSGI bundle where you are using Axis2 API.
7) If you ar using Maven, You can use this pom.xml dependencies

<!-- Axis2 Dependencies -->
<dependency>
<groupId>commons-codec</groupId>
<artifactId>commons-codec</artifactId>
<version>1.3</version>
</dependency>
<dependency>
<groupId>commons-httpclient</groupId>
<artifactId>commons-httpclient</artifactId>
<version>3.1</version>
</dependency>
<dependency>
<groupId>commons-logging</groupId>
<artifactId>commons-logging</artifactId>
<version>1.1</version>
</dependency>
<dependency>
<groupId>javax.activation</groupId>
<artifactId>activation</artifactId>
<version>1.1.1</version>
</dependency>
<dependency>
<groupId>javax.mail</groupId>
<artifactId>mail</artifactId>
<version>1.4</version>
</dependency>
<dependency>
<groupId>org.apache.axis2</groupId>
<artifactId>axis2-adb</artifactId>
<version>1.6.1</version>
</dependency>
<dependency>
<groupId>org.apache.axis2</groupId>
<artifactId>axis2-kernel</artifactId>
<version>1.6.1</version>
</dependency>
<dependency>
<groupId>org.apache.axis2</groupId>
<artifactId>axis2-transport-http</artifactId>
<version>1.6.1</version>
</dependency>
<dependency>
<groupId>org.apache.axis2</groupId>
<artifactId>axis2-transport-local</artifactId>
<version>1.6.1</version>
</dependency>
<dependency>
<groupId>org.apache.httpcomponents</groupId>
<artifactId>httpcore</artifactId>
<version>4.0</version>
</dependency>
<dependency>
<groupId>org.apache.neethi</groupId>
<artifactId>neethi</artifactId>
<version>3.0.1</version>
</dependency>
<dependency>
<groupId>org.apache.ws.commons.axiom</groupId>
<artifactId>axiom-api</artifactId>
<version>1.2.12</version>
</dependency>
<dependency>
<groupId>org.apache.ws.commons.axiom</groupId>
<artifactId>axiom-impl</artifactId>
<version>1.2.12</version>
</dependency>
<dependency>
<groupId>org.apache.ws.commons.schema</groupId>
<artifactId>XmlSchema</artifactId>
<version>1.4.7</version>
</dependency>
<dependency>
<groupId>stax</groupId>
<artifactId>stax</artifactId>
<version>1.2.0</version>
</dependency>
<dependency>
<groupId>stax</groupId>
<artifactId>stax-api</artifactId>
<version>1.0.1</version>
</dependency>
<dependency>
<groupId>wsdl4j</groupId>
<artifactId>wsdl4j</artifactId>
<version>1.6.2</version>
</dependency>

Please test it before use. Axis2 Integration is still work in progress .. I will update once have concrete result

How to change Admin password in CQ5.5

Use case Change admin password

Good news Till now you have to change admin password at multiple location if you are using CQ5.4, see here

From CQ5.5 onward you just have to change admin password once. There are multiple ways though,

1) Using CRX explorer (Old way)



2) Using granite http://host:port/libs/granite/security/content/admin.html#edit:/home/users/a/admin



3) using curl command

curl -u admin:OLD_PWD -F rep:password="NEW_PWD" http://host:port/home/users/a/admin.rw.html


You can also use security API for do user related task, Something like this

curl -d "createUser=true&authorizableId=$userId&rep:password=pw&membership=administrators" -u "admin:admin" http://host:port/libs/granite/security/post/authorizables

(There is set of operation you can do using this, JAVA DOC coming soon)


For any other user other than admin you can use CURL to change password as

curl rep:password="test" --user admin:ADMIN_PASSWORD http://host:port/home/users/a/aparker@geometrixx.com


You still have to use CURL command to change replication agent password mention here

For CQ5.6:

Above curl command does not work for CQ 5.6 instead use this command to change admin password

curl -u admin:<OLD PASSWORD> -Fplain=<NEW PASSWORD> -Fverify=<NEW PASSWORD>  -Fold=<OLD PASSWORD> -FPath=/home/users/a/admin http://HOST:PORT/crx/explorer/ui/setpassword.jsp

BELOW DOES NOT WORK IN CQ5.6 thats why have strike. Looking in to this.

You can also try http://sling.apache.org/documentation/bundles/managing-users-and-groups-jackrabbit-usermanager.html#change-password to change password of a USER. For that you first have to download http://mvnrepository.com/artifact/org.apache.sling/org.apache.sling.jcr.jackrabbit.usermanager/2.2.0 and then install it using felix console or through your code.

Note For all cases you need current admin password to perform operation. As always please test it before use.

Thanks Alex from Adobe for Helping.

How to expose a class through JMX in CQ5.5 / WEM

Use case: Starting with CQ5.5 / CRX2.3, JMX has eventually been integrated to expose various repository related statistics and information. There are also certain operations that can be triggered via JMX, such as online backup or TarPM optimization.

The Felix Console comes with a dedicated JMX plugin to access the JMX interface.


You can use any JMX client (Visual VM, Jconsole) to perform these operation, Here is screen shot of how it will look like



You can also perform JMX related operation through felix console




The main 2 classes that provide JMX information and data in CRX are as follows:
com.day.crx.sling.server.jmx.ManagedRepositoryMBean
com.day.crx.sling.server.impl.jmx.ManagedRepository
The former is an interface which defines the methods being exposed to JMX. These methods can be either read-only, so purely informative or writeable as well in which case e.g. configuration settings can be adjusted during runtime.

Creating custom JMX classes

Creating custom so called MBean classes is also quite straightforward. In the end, such classes are plain OSGi components that need to declare the javax.management.DynamicMBean interface, these are then automatically discovered during deployment and registered with JMX.

Here is sample class

package com.adobe.test;

import com.adobe.granite.jmx.annotation.Description;

@Description("JMX MBean example")
public interface JMXTestMBean {
@Description("Get bean name")
public String getName();
@Description("Get Bean Id")
public String getId();
@Description("Get repository Description")
public String getRepositoryName();
}


And implementation

package com.adobe.test.impl;

import javax.management.MBeanAttributeInfo;
import javax.management.MBeanInfo;
import javax.management.NotCompliantMBeanException;
import javax.management.StandardMBean;

import org.apache.felix.scr.annotations.Component;
import org.apache.felix.scr.annotations.Property;
import org.apache.felix.scr.annotations.Reference;
import org.apache.felix.scr.annotations.ReferenceCardinality;
import org.apache.felix.scr.annotations.Service;
import org.apache.sling.jcr.api.SlingRepository;

import com.adobe.test.JMXTestMBean;

@Component(immediate=true)
@Property(name="jmx.objectname", value="com.adobe.test:type=test")
@Service(value=javax.management.DynamicMBean.class)
public class JMXTestMBeanImpl extends StandardMBean implements JMXTestMBean {

@Reference(cardinality = ReferenceCardinality.OPTIONAL_UNARY)
private SlingRepository repository;

public JMXTestMBeanImpl() throws NotCompliantMBeanException {
super(JMXTestMBean.class);
}

@Override
protected String getDescription(MBeanInfo arg0) {
return "MBean Test example";
}

@Override
protected String getDescription(MBeanAttributeInfo mBeanAttributeInfo) {
return "MBean Attribute";
}

public String getName() {
// TODO Auto-generated method stub
return "Test JMX Bean";
}

public String getId() {
// TODO Auto-generated method stub
return "v0.1";
}

public String getRepositoryName(){
return repository.getDescriptor("jcr.repository.name");
}

}








You can get entire package from here

JMX API in CQ: http://dev.day.com/docs/en/cq/current/javadoc/com/adobe/granite/jmx/annotation/package-summary.html