How to Integrate 3rd party Jar file in CQ / WEM

Use Case: You have a non osgi jar file that you want to integrate in CQ as a bundle. You want to create fragment bundle (See option 4)

Solution: There are various way you can do this, Here is some options.

Option 1:

http://dev.day.com/content/kb/home/cq5/Development/ApacheFelix/ConvertAJarIntoOsgiBundle.html

jar cvfm <Your bundle name> manifest.txt <Non OSGI Jar file>

manifest.txt file will look like this

Manifest-Version: 1.0
Created-By: yogesh
Bundle-ManifestVersion: 2
Bundle-Name: <Your Bundle name>
Bundle-Description: JTMB bundle for dvdRental
Bundle-Version: 4.4.0
Bundle-ClassPath: .,<Your jar file>
Bundle-SymbolicName: <Bundle Symbolic name>
Export-Package: <Package you want to export, Just export selective package>
Import-Package: <Package this jar file is importing from others, You can put * as well>


After bundle is created you can put them in /apps/<your app>/install folder, So that they can be picked by Felix. Once in Install Folder you will see them in felix console

Option 2: (Only till CQ5.4)

Other option is use sling boot delegation http://dev.day.com/content/kb/home/cq5/Development/SlingBootdelegation.html to add third party jar.


Option 3: 

Using BND tool

http://help.adobe.com/en_US/enterpriseplatform/10.0/AEPDeveloperGuide/WS562be5d616a63050-c62634713136615f38-8000.html

1) Install bnd.jar
2) Use command java -jar bnd.jar wrap {YourJarfile}.jar
3) Rename .bar file to {file-name}-bundle.jar
4) Put it in install folder or install using felix console

Create OSGI bundle using BND with additional property

java -jar bnd.jar wrap -properties bundle.bnd {YourJarfile}.jar
bundle.bnd will look something like this
version=<Version>
Export-Package: <Package you want to export>
Private-Package: <Package you want to import>
Bundle-Version: 1
Bundle-Classpath: .,{yourjarfile}

More information of available options here http://fusesource.com/docs/esb/4.2/deploy_osgi/DeployJar-Convert.html

Option 4:

Using maven-bundle-plugin

1) Put your Jar file under /libs folder (using Web Dav) of your project and include it in project menifest or pom.xml

If you are using CRXDE then your bundle menifest will look like this. In this example, I assume that /libs folder is created at same level of your menifest file.

Export-Package: <Package you want to export, Use * for everything but not recommended>
Import-Package: *;resolution:=optional
Private-Package: <If you want>
Dynamic-Import: *
Embed-Dependency: *;scope=compile|runtime
Embed-Directory: /libs
Embed-Transitive: true

Your pom.xml will look like this

<build>
        <plugins>
        <plugin>
                        <groupId>org.apache.felix</groupId>
                        <artifactId>maven-bundle-plugin</artifactId>
                        <version>2.0.1</version>
                        <extensions>true</extensions>
<!-- You can use extension true to create fragment bundle -->

                <configuration>
                 
                    <instructions>
                                     <Bundle-SymbolicName>${project.groupId}.${project.artifactId}</Bundle-SymbolicName>
                                        <Bundle-Version>${project.version}</Bundle-Version>
                                        <Include-Resource>lib=${basedir}/lib</Include-Resource>
                                        <Bundle-ClassPath>
                                                .,
                                               .... All non OSGI jar file
                                        </Bundle-ClassPath>
                                        <Fragment-Host>system.bundle; extension:=framework</Fragment-Host>
                                        <Export-Package>
                                           ... Your comma seperated Export Package
                                        </Export-Package>
                                        <Import-Package>!*</Import-Package>
                                        <Embed-Dependency>*;scope=compile|runtime</Embed-Dependency>
                                        <Embed-Transitive>true</Embed-Transitive>
                        </instructions>
                        </configuration>
                </plugin>
        </plugins>
    </build>

Some more useful doc

http://www.cqblueprints.com/xwiki/bin/view/Blue+Prints/Deploying+3rd+Party+Libraries

http://felix.apache.org/site/apache-felix-maven-bundle-plugin-bnd.html


A good example is http://www.wemblog.com/2012/03/how-to-integrate-soap-web-service.html

Feel free to let me know if missing something.

How to install Multiple Package in CQ / WEM

Use Case: You have a Hotfix that has many dependencies and you want to install them in order Or you want all packages to install in particular order. Installing through package manager can be time consuming process.

Solution:

When CQ starts up it checks /crx-quickstart/repository/install till CQ5.4 

and /crx-quickstart/install CQ5.5 and above use  folder for any new package and install them in alphabetical order.

So for example you have a package which has many dependencies, You can rename them in alphabetical order and put them in /crx-quickstart/repository/install folder. Next CQ restart will make sure that they are installed in that order.

For example, If you have Zip file name p001-hotfix-<Some-number>.zip and p002-hotfix-<some-number> under install folder, then p001 will get installed first and then p002 upon restart. After initial installation CQ creates properties file (some thing like <your-package-name>.properties) associated with each package, It is used by installer as a indicator not to install that package again on next restart.

You can also mention start order of package by creating start level folder. For example to assign startlevel 21 to bundle xyz you place the file into crx-quickstart/install/21. The default start level is assigned if you place it into the folder called 0 or directly into the install folder. This is also true for any bundle you put in /install folder in repository (Which is picked up by jcr installer as soon you put them there). Note that number might have preference over alphabets, so rename your files accordingly.

You have to restart CQ after putting files in install folder.

This approach does not work for CRX Hotfix.

Few more facts about packages that is worth mentioning is,

1) If you activate a package from author instance to publish, They get "INSTALLED" on publish instance. In CQ5.5 you have replicate option on package manager it self

For CQ5.4 you have to go to tools->packages-><Your package location> and then click on activate

This is very useful if you have many publish instance and you want to install package to all of them. This is also useful when you have to move package between environments.

2) In cluster instance you only have to install package in one instance (It can be any one, But master is recommended)

I have seen many clients do not know about these features so thought to mention it. Please feel free to comment if you have any question about it.

Caution: Note that, If you use /crx-quickstart/repository/install to install your Hotfix, You might not be able to uninstall it, As no snapshot of that package is created in that case. So if you are testing multiple Hotfix you might want to use package manager in that case.

How to deploy portlet in CQ5.5

Use Case: You want to deploy portet in CQ5.5

Prerequisite: http://dev.day.com/docs/en/cq/current/administering/cq_as_portal_54.html Above
document covers most of the part. I will cover what is changed in CQ5.5

There is no servlet engine UI. So you can not deploy your portlet application using Day servlet engine any more.

 Here is steps to deploy portlet war file in CQ5.5

 1) Chose any servlet engine of your choice. I am taking example of tomcat in this case
 2) Deploy your CQ war file using instruction given in http://dev.day.com/docs/en/cq/current/howto/install_application_server.html
 3) Install package given in /opt/portal/cq-portlet-components.zip using CQ package manager
 4) Using webDav copy /libs/portal/director/cq-portlet-director-sharedlibs.*.jar in put it in your shared lib directory of your servlet container (I put it in /libs folder of tomcat) 
5) Now take cq-portlet-webapp-*.war (This is just an example war for OOTB portlet) and deploy it in your servlet container (In my case I just put it in webapps folder of tomcat)
6) After deploying make sure that it is accesible using http://HOST:PORT/cqportlet/cqbridge/system/console
7) Now follow instruction in http://dev.day.com/docs/en/cq/current/administering/cq_as_portal_54.html#Configuring the Portlet and give your CQ end point.
8) Now go to web.xml of your portlet war file and add following entry

    <servlet-name>slingportal</servlet-name>
     <servlet-class>org.apache.sling.portal.container.api.ContainerServlet</servlet-class>
        <load-on-startup>1</load-on-startup>
    </servlet>
    <servlet-mapping>
        <servlet-name>slingportal</servlet-name>
        <url-pattern>/SlingPortletInvoker</url-pattern>
    </servlet-mapping>

9) Then add portlet component and test as given in http://dev.day.com/docs/en/cq/current/administering/cq_as_portal_54.html#Using CQ as a Portal

Portlet support is very limited in CQ5.5, So unless it is absolutely required, do not use it.

Note: As always, Please test before using this to production 

How to change package install behavior in CQ / WEM

Use case: Suppose you want to move some content using CQ package, from one instance to another instance. Now you want to have a behavior where content on destination don't get deleted if it is not there in source package.
In order to better understand the problem, Consider following example,
On source you have following structure
/content/.../en
/content/.../en/p1
/content/.../en/p2
/content/.../en/p3
/content/.../en/p5
And on destination you have following structure
/content/.../en
/content/.../en/p1
/content/.../en/p2
/content/.../en/p3
/content/.../en/p4
Now when you will create package using filter root as /content/.../en on source and deploy that package on destination, You would end up deleting node /content/.../en/p4 on destination, as it is not part of source package.

On package manager through UI there is no way you can specify what behavior you want for that package on destination (You want something similar to AC handling behavior). Here is something you can do to override this behavior.

Solution
Consider your package name as testPackage.zip
Then do following
mkdir foo
cd foo
cp testPackage.zip foo/
jar xvf testPackage.zip
cd META-INF/vault
vi filter.xml
Add this to root def <filter root="/content" mode="merge">
cd ../..
jar -cvfM <New Package Name> META-INF/ jcr_root/
Now free to use <New Package> with new behavior

There are three Mode available,
1) replace: Normal behavior. Existing content is replaced completly by the imported content, i.e. is overridden or deleted accordingly.
2) merge: Existing content is not modified, i.e. only new content is added and none is deleted or modified
3) update: Existing content is only updated but never deleted

If you are using API to make changes then it is available here com.day.jcr.vault.fs.api.ImportMode

Well .. If you see you just created a package using file system content using simple Java package tool. Yes you can absolutely do that as long as you maintain directory structure of package i.e. you need to have following files in package (Take an example from any existing package created using package manager)

|-- META-INF/
| `-- vault/
| |-- config.xml
| |-- settings.xml
| |-- filter.xml
| |-- properties.xml
`-- jcr_root/
.
.

filter.xml
contains a serialization of the filters defined in the filters node of the package in a default file vault manner.
properties.xml
contains additional package properties like creation date and user.
nodetypes.cnd
contains a cnd serialization of all nodetypes and transitive ones used in the package.
config.xml
contains the file vault config that was present when exporting.

If you are using maven build, You can use Day file Vault Maven plugin to create package as part of your maven deployment. Your pom.xml will have entry like this


<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd"
>
<modelVersion>4.0.0</modelVersion>

<groupId>com.day.jcr.vault</groupId>
<artifactId>vault.test</artifactId>
<version>0.0.1-SNAPSHOT</version>
<packaging>content-package</packaging>

<name>Sample Content Package Project</name>
<description>
Project to test the Maven Vault Plugin functionality
</description>

<build>
<plugins>
<plugin>
<groupId>com.day.jcr.vault</groupId>
<artifactId>maven-vault-plugin</artifactId>
<version>0.0.10</version>
<extensions>true</extensions>
</plugin>
</plugins>
</build>
</project>

If you want to create package using maven plug in you have to use resource, Something like

<build>

 <resources>

<resource>
<directory>${basedir}/src/main/content/META-INF</directory>
<targetPath>../vault-work/META-INF</targetPath>
</resource>

 <resource>

 <directory>${basedir}/jcr_root</directory>

 <targetPath>.</targetPath>

 <filtering>true</filtering>

 <includes>

<include>

</include>

 </includes>

  <excludes>

       <exclude>**/.vlt</exclude>

       <exclude>**/.vltignore</exclude>

       <exclude>**/.DS_Store</exclude>

       </excludes>

 </resource>

 </resources>

And then run mvn <goal>

Note that your packaging type should be content-package. End result would be CQ/CRX package with  all filters in resource section.


Note: In this case you have to manage both filter.xml (Under META-INF) and <resource> tag under pom.xml for filter definition. <resource> tag under pom.xml is used by maven valut plugin to include resource in content package and filter.xml is used by package manager to include resource. There is already a enhancement request to merge these two and scheduled by later release of vault plugin.

Here is some more info

http://ruben42.wordpress.com/2012/03/22/maven-vault-plugin/

http://ben-peter.com/blog/generating-aggregate-content-packages-for-crx/

Vault API in CQ can be found here: http://dev.day.com/docs/en/cq/current/javadoc/overview-summary.html

Special Thanks for tobias Bocanegra from Adobe for helping to understand some of stuff

How to Use LDAP with CQ / AEM

Use case: Configure LDAP with CQ / AEM

Why There are some changes in LDAP configuration in CQ5.5 and documetation is still in progress. Before actual documentation comes, Here is few things you should know

Pre requisite Keep http://dev.day.com/docs/en/cq/current/core/administering/ldap_authentication.html on side. I will mention things changed.

No sample_ldap_login.conf under crx-quickstart\server\etc : In fact there is no crx-quickstart\server\etc. You can put your sample_ldap_login.conf any where you want. You can get sample ldap.conf from http://dev.day.com/content/kb/home/cq5/CQ5SystemAdministration/LdapConfig.html or here

2) Once downloaded you can put this file to any location (I prefer /crx-quickstart/conf) and then change your start script to point to this location.

3) I have seen, LDAP not starting because of some wrong configuration in repository.xml. Also attaching sample repository.xml that you can use directly in CQ5.5 / CRX2.3

How to enable logging for LDAP in CQ5.5 / CQ5.6
1) Note that LDAP module is now deployed as OSGI module. You have to follow following step to enable logging for LDAP

[1] Log into Felix Console: http://<host>:<port>/system/console/configMgr
[2] From "Factory Configurations", create "Apache Sling Logging Writer Configuration"
[2.1] Set value of "Log File" to "../logs/ldap.log"
[2.2] Click on "Save"
[3] From "Factory Configurations", create "Apache Sling Logging Logger Configuration"
[3.1] Set value of "Log Level" to "Debug"
[3.2] Set value of "Log File" to "../logs/ldap.log"
[3.3] Add "Logger" => com.day.crx.security.ldap
[3.4] Click on "Save"

Step [2] is require for log rotation, If you don't want to rotate log then skip that step.

user purge and sync option In CQ5.5 GA release, there is no user purge and sync option. This is moved to JMX. This option is now available as CQ5.5 Update Package 1, That you can download it from package share. To use this option, Please install CQ5.5 Update 1, Start CQ with LDAP enable and then go to Felix JMX console. You would see something like this

Full path in JVM Param I have seen issues with some customer where using relative path for ldap.conf file does not work. Please use complete path of file system instead.

Change in start script: CQ5.5 OOTB has options enabled for jass config. You can use following environment variable to enable LDAP through start script

CQ_USE_JAAS='true'
CQ_JAAS_CONFIG='< Complete Path to LDAP conf file>'

See how it is used in start script

if [ $CQ_USE_JAAS ]; then
START_OPTS="${START_OPTS} -Djava.security.auth.login.config=${CQ_JAAS_CONFIG}"

Secure LDAP Password:

http://helpx.adobe.com/experience-manager/kb/encrypted-password-jaas-config.html

One more interesting Blog http://www.citytechinc.com/us/en/blog/2012/03/adobe_cq5_5_and_ldap.html

Hope fully all changes are covered. Shoot me any question if it is not clear for LDAP with CQ5.5.


Configure LDAP in CQ / AEM6

In CQ6 All login module can be configured as Pluggable Login Module. What that mean is there is no need to create JAAS config through start up param. You can configure your LDAP using OSGI config. Steps of how to do that can be found from

http://docs.adobe.com/docs/en/aem/6-0/administer/security/ldap-config.html

How to create OSGI config can be found from here http://www.wemblog.com/2012/10/how-to-work-with-configurations-in-cq.html

this is how sample LDAP config will look like in AEM6 (Here custom can be any thing)

/apps
 /yourcompany
   /configurations
       /config.author
         /org.apache.jackrabbit.oak.security.authentication.ldap.impl.LdapIdentityProvider-custom.xml
         /org.apache.jackrabbit.oak.spi.security.authentication.external.impl.DefaultSyncHandler-custom.xml
         /org.apache.jackrabbit.oak.spi.security.authentication.external.impl.ExternalLoginModuleFactory-custom.xml


Here is how content of these file look like,

1) org.apache.jackrabbit.oak.security.authentication.ldap.impl.LdapIdentityProvider-custom.xml

There are more attribute as well please read above documentation for that.

<?xml version="1.0" encoding="UTF-8"?>

<jcr:root xmlns:sling="http://sling.apache.org/jcr/sling/1.0" xmlns:jcr="http://www.jcp.org/jcr/1.0"

    jcr:primaryType="sling:OsgiConfig"

    provider.name="custom_ldap"

    host.name="server-name"

    host.port="server-port"

    bind.dn="same as authDn"

    bind.password="same as password"

    user.baseDN="same as userRoot"

    user.idAttribute="same as userIdAttribute"

    user.extraFilter="same as userFilter"

    group.baseDN="same as groupRoot"

    group.extraFilter="same as groupFilter"/>

2) org.apache.jackrabbit.oak.spi.security.authentication.external.impl.DefaultSyncHandler-custom.xml

following property could defer based on your LDAP.

<?xml version="1.0" encoding="UTF-8"?>

<jcr:root xmlns:sling="http://sling.apache.org/jcr/sling/1.0" xmlns:jcr="http://www.jcp.org/jcr/1.0"

    jcr:primaryType="sling:OsgiConfig"

    handler.name="custom_ldap_handler"

    user.expirationTime="1d"

    user.autoMembership="[any CQ group]"

    user.propertyMapping="[rep:email=mail,rep:fullName=cn,profile/email=mail,profile/familyName=sn,profile/givenName=gn]"

    user.membershipExpTime="1d"

    group.propertyMapping="[description=description,rep:email=mail,rep:fullname=cn]"

    />

3) org.apache.jackrabbit.oak.spi.security.authentication.external.impl.ExternalLoginModuleFactory-custom.xml

<?xml version="1.0" encoding="UTF-8"?>

<jcr:root xmlns:sling="http://sling.apache.org/jcr/sling/1.0"

  xmlns:jcr="http://www.jcp.org/jcr/1.0" 

  jcr:primaryType="sling:OsgiConfig"

  jaas.controlFlag="REQUIRED"

  idp.name="custom_ldap" 

  sync.handlerName="custom_ldap_handler" />