Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Issues (to be solved before the release)

NumberIssueFix

DD4T.Providers.DxaModelService DLL is built with dependency on DLLs that are not added to the dependencies section in the Nuspec. Dependencies used in the DLL, are:

  • DD4T.ContentModel 2.2.1
  • DD4T.ContentModel.Contracts 2.2.1 (I installed Nuget version 2.2.2)
  • DD4T.Core.Contracts 2.2.2 (I installed Nuget version 2.2.7)
  • DD4T.Utils 2.2.2 (I installed Nuget version 2.2.7)
  • Tridion.ContentDelivery 10.1.0.1003 (Nuget version 10.1.0)
  • Sdl.Web.Delivery.Caching 10.1.0.1003 (Nuget version 10.1.0)

This should be reflected in the Nuspec so they are installed automatically. The result right now is a type loading exception. 

Workaround is to manually configure bindings in the Web.config, but this shouldn't be necessary.

ASP.NET MVC presents the URL to the action method WITHOUT a leading slash (so 'en/index.html' instead of '/en/index.html'). In DD4T, the slash is not prepended to the URL in the framework itself. Instead, it is normally done in the concrete controller implementation (in other words: the customer knows they have to add the leading slash themselves).

The DXA 2 model service provider (I think it is in the provider, could be in the client as well) prepends the slash to the URL itself. As a result, all existing implementations return a 404, because the URL that is looked up in the broker, is '//en/index'.

Fixed by checking if the URL starts with a single slash, and only prepend a slash if it doesn't.

Page cannot be deserialized when JSON is generated by DXA R2 TBBs. The error thrown in the application is 'Value cannot be null, parameter name: source'.

Image Removed

The line that causes the error, is made red below (from the ContentModels.cs class file in DD4T.Model):

[XmlIgnore]
IList<IComponentPresentation> IPage.ComponentPresentations
{
get { return ComponentPresentations.ToList<IComponentPresentation>(); }
}

Looking at the JSON generated by the model service, I think it is simple: the conversion to DD4T-style JSON is not happening. The DD4T and DXA2 models are the same for the top level properties of the page (like Id and Title) but fail on the ComponentPresentations element, which does not exist on that level in DXA2.

Update QS 23 Jan: the root cause has been identified by Alexey: it fails if the navigation.json page is not published. Since the point of this compatibility is to support existing DD4T customers who want to profit from the model service, it makes no sense to force them to create this page and publish it (they probably have an alternative navigation mechanism in place already). We need a version of the model service that does not fail when the navigation.json page is missing.

Update QS 23 Jan (later on the day): the process also needs schemas.json. Bart is recommending to publish the Publish Settings page, which in turn publishes the above-mentioned JSON files as BinaryVariants. We will do that and retest.

Update Jan 25: When running a dd4t-2-java app against http://ec2-52-17-130-35

Links are resolved in the model service. This makes it impossible for customers to write their own linking logic, which many of them have done. Please make this a configurable feature and default to false.

DXA publishes rich text like this:

<p>Hello <a href="tcm:90-123">world</a></p>

It resolves this into:

<p>Hello <a href="/world">world</a></p>

Although this is great, DD4T has link resolvers in the application which expect xlink:href attributes. The rich text should therefore be resolved into this:

<p>Hello <a href="/world" xlink:href="tcm:90-123" xmlns:xlink="http://www.w3.org/1999/xlink">world</a></p>

NOTE: latest version allows you to chose whether or not to resolve links AND remove namespaces. SDL is still discussing internally whether to split these to settings into:

  1. Resolve links (true or false, if true adds href="URL" without a namespace)
  2. Remove XMLNS (true or false, if true removes all namespace declarations and all xlink:* attributes)

When you start the model service before the discovery and content services are started, it returns the following error (even after the other services have been started):

{
"timestamp": "2018-01-29T17:13:22.830+0000",
"status": 500,
"error": "Internal Server Error",
"exception": "com.google.gson.JsonSyntaxException",
"message": "com.google.gson.stream.MalformedJsonException: Invalid escape sequence at line 1 column 1194 path $.claimValues..DOCUMENT_ROOT",
"path": "/PageModel/tcm/2//dd4t.html"
}

This may not be high prio or even a real issue, as long as there is a dependency configured from the model service to the discovery service. (QS)

The model service resolves links in rich text as well as the LinkUrl property in the DXA model, into hyperlinks without an extension. I understand why, because it is clean and many customers want it that way. But there are customers who currently route only **.html to their DD4T page controller. For these customers, the links will not work.

Could you make this behavior configurable? E.g.:

dxa.defaults.converter-extensionless-urls=true|false

It can default to true, so the default behavior remains the same as in the CTP.

NumberPrioIssue
2 3 TSI-30143034

TSI-3032

Medium3 - TSI-3034Low
8 - TSI-3024HighThere should be no copies of the legacy DD4T libraries in the DXA codebase. This is confusing to the customers and does not have any real advantages. We need to align releases (and hopefully not release DD4T at all anymore, other than defect fixes).
9 - TSI-3033High

Fixed issues

Low

When you start the model service before the discovery and content services are started, it returns the following error (even after the other services have been started):

{
"timestamp": "2018-01-29T17:13:22.830+0000",
"status": 500,
"error": "Internal Server Error",
"exception": "com.google.gson.JsonSyntaxException",
"message": "com.google.gson.stream.MalformedJsonException: Invalid escape sequence at line 1 column 1194 path $.claimValues..DOCUMENT_ROOT",
"path": "/PageModel/tcm/2//dd4t.html"
}


This may not be high prio or even a real issue, as long as there is a dependency configured from the model service to the discovery service. (QS)

8 - TSI-3024HighThere should be no copies of the legacy DD4T libraries in the DXA codebase. This is confusing to the customers and does not have any real advantages. We need to align releases (and hopefully not release DD4T at all anymore, other than defect fixes).


Fixed issues

Model conversion (to R2) fails if page template has no metadata (which is a valid scenario in DD4T). 

The error is in the class ToR2ConverterImpl, in the _loadIncludes method. Please check if the pageTemplate has metadata at all before checking if it contains a certain field.

NumberIssueFix

DD4T.Providers.DxaModelService DLL is built with dependency on DLLs that are not added to the dependencies section in the Nuspec. Dependencies used in the DLL, are:

  • DD4T.ContentModel 2.2.1
  • DD4T.ContentModel.Contracts 2.2.1 (I installed Nuget version 2.2.2)
  • DD4T.Core.Contracts 2.2.2 (I installed Nuget version 2.2.7)
  • DD4T.Utils 2.2.2 (I installed Nuget version 2.2.7)
  • Tridion.ContentDelivery 10.1.0.1003 (Nuget version 10.1.0)
  • Sdl.Web.Delivery.Caching 10.1.0.1003 (Nuget version 10.1.0)

This should be reflected in the Nuspec so they are installed automatically. The result right now is a type loading exception. 

Workaround is to manually configure bindings in the Web.config, but this shouldn't be necessary.



ASP.NET MVC presents the URL to the action method WITHOUT a leading slash (so 'en/index.html' instead of '/en/index.html'). In DD4T, the slash is not prepended to the URL in the framework itself. Instead, it is normally done in the concrete controller implementation (in other words: the customer knows they have to add the leading slash themselves).

The DXA 2 model service provider (I think it is in the provider, could be in the client as well) prepends the slash to the URL itself. As a result, all existing implementations return a 404, because the URL that is looked up in the broker, is '//en/index'.

Fixed by checking if the URL starts with a single slash, and only prepend a slash if it doesn't.

Page cannot be deserialized when JSON is generated by DXA R2 TBBs. The error thrown in the application is 'Value cannot be null, parameter name: source'.


Image Added

The line that causes the error, is made red below (from the ContentModels.cs class file in DD4T.Model):

[XmlIgnore]
IList<IComponentPresentation> IPage.ComponentPresentations
{
get { return ComponentPresentations.ToList<IComponentPresentation>(); }
}

Looking at the JSON generated by the model service, I think it is simple: the conversion to DD4T-style JSON is not happening. The DD4T and DXA2 models are the same for the top level properties of the page (like Id and Title) but fail on the ComponentPresentations element, which does not exist on that level in DXA2.

Update QS 23 Jan: the root cause has been identified by Alexey: it fails if the navigation.json page is not published. Since the point of this compatibility is to support existing DD4T customers who want to profit from the model service, it makes no sense to force them to create this page and publish it (they probably have an alternative navigation mechanism in place already). We need a version of the model service that does not fail when the navigation.json page is missing.

Update QS 23 Jan (later on the day): the process also needs schemas.json. Bart is recommending to publish the Publish Settings page, which in turn publishes the above-mentioned JSON files as BinaryVariants. We will do that and retest.


Update Jan 25: When running a dd4t-2-java app against http://ec2-52-17-130-35.eu-west-1.compute.amazonaws.com:8081/content.svc, it seems that the ModelService doesnt convert PageTemplate info. We need to decide whether this is a blocking issue or not. Quirijn Slings - we first also need to know how this page was published (with DXA or DD4T TBBs)

Update Jan 30: everything works if a. you have published the publish settings page and the navigation page, and b. you have assigned DXA metadata to your DXA page template. This is something that must be made clear in the release notes.



Access any page, for example
and 500 error is returned.
In the site.log file, errors related to the model service are found and a URL it is trying to call.
The errors are always similar to:
DXA Model Service returned an error response: Status=500, Error='Internal Server Error', Exception='java.lang.NullPointerException', Message='No message available', Path='/PageModel/tcm/16/further-information'
Copy the model service URL found in the log file and request it in the browser.
A perfectly normal JSON is retrieved.
Access again
and now it works great.
When trying to access any other page, the error cycle appears. 
View file
nameerror dxa.txt
height250
Update KS: Tested on a fresh 8.5 instance with the latest hotfix CD_8.5.0.8403. The same results are observed as the last time.
The only difference is that this time cachine was left as default (web.config of the web application wasn't changed at all) so that returning to the previous page did not return 500 since it was probably retrieved from the cache.
Still, the main issue is there: Requesting of the new pages does not work unless the model service is not requested directly and via the web application. 
Update KS: Issue fix after the new version of DXA app was delivered by SDL. 


When using "Generate DXA R2 Page Model", the metadata of the page is not included in the JSON that is returned by the model service. Sorry, apparently there is a Meta property and a Metadata property and I just got confused.
5Fatal5Fatal

Model conversion from R2 to DD4T fails.

I request a page containing DXA R2 JSON and publish it with URL /dxa.html. The JSON looks like this (when retrieved with raw=true): dxa-raw.json.

I then try to convert it to DD4T by calling the model service with this url:

http://ec2-52-214-122-181.eu-west-1.compute.amazonaws.com:

8081/content.svc, it seems that the ModelService doesnt convert PageTemplate info. We need to decide whether this is a blocking issue or not. Quirijn Slings - we first also need to know how this page was published (with DXA or DD4T TBBs)

Update Jan 30: everything works if a. you have published the publish settings page and the navigation page, and b. you have assigned DXA metadata to your DXA page template. This is something that must be made clear in the release notes.

Access any page, for example
and 500 error is returned.
In the site.log file, errors related to the model service are found and a URL it is trying to call.
The errors are always similar to:
DXA Model Service returned an error response: Status=500, Error='Internal Server Error', Exception='java.lang.NullPointerException', Message='No message available', Path='/PageModel/tcm/16/further-information'
Copy the model service URL found in the log file and request it in the browser.
A perfectly normal JSON is retrieved.
Access again
and now it works great.
When trying to access any other page, the error cycle appears. 
View file
nameerror dxa.txt
height250
Update KS: Tested on a fresh 8.5 instance with the latest hotfix CD_8.5.0.8403. The same results are observed as the last time.
The only difference is that this time cachine was left as default (web.config of the web application wasn't changed at all) so that returning to the previous page did not return 500 since it was probably retrieved from the cache.
Still, the main issue is there: Requesting of the new pages does not work unless the model service is not requested directly and via the web application. 
Update KS: Issue fix after the new version of DXA app was delivered by SDL. 
When using "Generate DXA R2 Page Model", the metadata of the page is not included in the JSON that is returned by the model service. Sorry, apparently there is a Meta property and a Metadata property and I just got confused.

Model conversion from R2 to DD4T fails.

I request a page containing DXA R2 JSON and publish it with URL /dxa.html. The JSON looks like this (when retrieved with raw=true): dxa-raw.json.

I then try to convert it to DD4T by calling the model service with this url:

http://ec2-52-214-122-181.eu-west-1.compute.amazonaws.com:8089/PageModel/tcm/2/dxa?modelType=DD4T

This results in this error:

{
"timestamp": "2018-01-30T13:18:29.967+0000",
"status": 500,
"error": "Internal Server Error",
"exception": "java.lang.NullPointerException",
"message": "No message available",
"path": "/PageModel/tcm/2/dxa"
}

The dxa log of the model service shows this error:

2018-01-30 13:18:29,943 INFO c.s.d.m.s.DefaultPageModelService - Found R2 model while requested DD4T, need to process R2 and convert, request PageRequestDto(publicationId=2, uriType=tcm, path=/dxa, includePages=INCLUDE, contentType=MODEL, dataModelType=DD4T, expansionDepth=100, depthCounter=8089/PageModel/tcm/2/dxa?modelType=DD4T

This results in this error:

{
"timestamp": "2018-01-30T13:18:29.967+0000",
"status": 500,
"error": "Internal Server Error",
"exception": "java.lang.NullPointerException",
"message": "No message available",
"path": "/PageModel/tcm/2/dxa"
}


The dxa log of the model service shows this error:

2018-01-30 13:18:29,943 INFO c.s.d.m.s.DefaultPageModelService - Found R2 model while requested DD4T, need to process R2 and convert, request PageRequestDto(publicationId=2, uriType=tcm, path=/dxa, includePages=INCLUDE, contentType=MODEL, dataModelType=DD4T, expansionDepth=100, depthCounter=com.sdl.dxa.common.dto.DepthCounter@9f)
2018-01-30 13:18:29,953 DEBUG o.s.b.f.s.DefaultListableBeanFactory - Returning cached instance of singleton bean 'cacheManager'
2018-01-30 13:18:29,953 DEBUG o.s.b.f.s.DefaultListableBeanFactory - Returning cached instance of singleton bean 'cacheManager'
2018-01-30 13:18:29,954 DEBUG o.s.b.f.s.DefaultListableBeanFactory - Returning cached instance of singleton bean 'cacheManager'
2018-01-30 13:18:29,954 DEBUG o.s.w.s.m.m.a.ExceptionHandlerExceptionResolver - Resolving exception from handler [public org.springframework.http.ResponseEntity com.sdl.dxa.modelservice.controller.PageModelController.getPage(java.lang.String,int,com.sdl.dxa.common.dto.PageRequestDto$PageInclusion,com.sdl.dxa.common.dto.DataModelType,boolean,javax.servlet.http.HttpServletRequest) throws com.sdl.webapp.common.api.content.ContentProviderException,com.fasterxml.jackson.core.JsonProcessingException]: java.lang.NullPointerException
2018-01-30 13:18:29,954 DEBUG o.s.w.s.m.a.ResponseStatusExceptionResolver - Resolving exception from handler [public org.springframework.http.ResponseEntity com.sdl.dxa.modelservice.controller.PageModelController.getPage(java.lang.String,int,com.sdl.dxa.common.dto.PageRequestDto$PageInclusion,com.sdl.dxa.common.dto.DepthCounter@9f)
2018-01-30 13:18:29,953 DEBUG o.s.b.f.s.DefaultListableBeanFactory - Returning cached instance of singleton bean 'cacheManager' DataModelType,boolean,javax.servlet.http.HttpServletRequest) throws com.sdl.webapp.common.api.content.ContentProviderException,com.fasterxml.jackson.core.JsonProcessingException]: java.lang.NullPointerException
2018-01-30 13:18:29,953 955 DEBUG o.s.bw.s.fm.s.DefaultListableBeanFactory - Returning cached instance of singleton bean 'cacheManager'
2018-01-30 13:18:29,954 DEBUG o.s.b.f.s.DefaultListableBeanFactory - Returning cached instance of singleton bean 'cacheManager'
2018-01-30 13:18:29,954 DEBUG o.s.w.s.m.m.a.ExceptionHandlerExceptionResolver - Resolving exception from handler [public org.springframework.http.ResponseEntity com.sdl.dxa.modelservice.controller.PageModelController.getPage(DefaultHandlerExceptionResolver - Resolving exception from handler [public org.springframework.http.ResponseEntity com.sdl.dxa.modelservice.controller.PageModelController.getPage(java.lang.String,int,com.sdl.dxa.common.dto.PageRequestDto$PageInclusion,com.sdl.dxa.common.dto.DataModelType,boolean,javax.servlet.http.HttpServletRequest) throws com.sdl.webapp.common.api.content.ContentProviderException,com.fasterxml.jackson.core.JsonProcessingException]: java.lang.NullPointerException
2018-01-30 13:18:29,954 955 DEBUG o.s.w.s.m.a.ResponseStatusExceptionResolver - Resolving exception from handler [public org.springframework.http.ResponseEntity DispatcherServlet - Could not complete request
java.lang.NullPointerException: null
at com.sdl.dxa.modelservice.service.processing.controllerconversion.PageModelControllerToDd4tConverterImpl.getPage_buildPageTemplate(java.lang.String,int,ToDd4tConverterImpl.java:201)
at com.sdl.dxa.commonmodelservice.dtoservice.PageRequestDto$PageInclusion,comprocessing.sdlconversion.dxaToDd4tConverterImpl.common.dto.DataModelType,boolean,javax.servlet.http.HttpServletRequest) throws convertToDd4t(ToDd4tConverterImpl.java:137)
at com.sdl.dxa.modelservice.service.DefaultPageModelService._processDd4tPageModel(DefaultPageModelService.java:99)
at com.sdl.webappdxa.commonmodelservice.apiservice.content.ContentProviderException,com.fasterxml.jackson.core.JsonProcessingException]: java.lang.NullPointerException
2018-01-30 13:18:29,955 DEBUG o.s.w.s.m.s.DefaultHandlerExceptionResolver - Resolving exception from handler [public org.springframework.http.ResponseEntity com.sdl.dxa.modelservice.controller.PageModelController.getPage(java.lang.String,int,com.sdl.dxa.common.dto.PageRequestDto$PageInclusion,com.sdl.dxa.common.dto.DataModelType,boolean,javax.servlet.http.HttpServletRequest) throws com.sdl.webapp.common.api.content.ContentProviderException,com.fasterxml.jackson.core.JsonProcessingException]: java.lang.NullPointerException
2018-01-30 13:18:29,955 DEBUG o.s.w.s.DispatcherServlet - Could not complete request
java.lang.NullPointerException: null
at com.sdl.dxa.modelservice.service.processing.conversion.ToDd4tConverterImpl._buildPageTemplate(ToDd4tConverterImpl.java:201)
at com.sdl.dxa.modelservice.service.processing.conversion.ToDd4tConverterImpl.convertToDd4t(ToDd4tConverterImpl.java:137)
at com.sdl.dxa.modelservice.service.DefaultPageModelService._processDd4tPageModel(DefaultPageModelService.java:99)
at com.sdl.dxa.modelservice.service.DefaultPageModelService.loadLegacyPageModel(DefaultPageModelService.java:80)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:333)
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:190)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
at org.springframework.cache.interceptor.CacheInterceptor$1.invoke(CacheInterceptor.java:52)

Note that the DXA page template that we used on this page, has metadata assigned to it. 

4High
1LowXHTML namespaces are removed by the model service. This is probably not a problem for any customer, but it is a difference so we should put it in the release notes.
7 - TSI-3028HighImporting the DXA CMS-items into an existing BluePrint hierarchy which already has a configured Topology with discovery services and everything, is too hard. It can really only be done by importing too much and then rewiring the topology manually. We cannot assume that customers will figure this out. 
Update: it is really much easier now with the new verison of the master-only.zip! The scenario should be described in the documentation, but it looks good now

DefaultPageModelService.loadLegacyPageModel(DefaultPageModelService.java:80)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:333)
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:190)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
at org.springframework.cache.interceptor.CacheInterceptor$1.invoke(CacheInterceptor.java:52)

Note that the DXA page template that we used on this page, has metadata assigned to it. 

4High

Model conversion (to R2) fails if page template has no metadata (which is a valid scenario in DD4T). 


The error is in the class ToR2ConverterImpl, in the _loadIncludes method. Please check if the pageTemplate has metadata at all before checking if it contains a certain field.

1LowXHTML namespaces are removed by the model service. This is probably not a problem for any customer, but it is a difference so we should put it in the release notes.
7 - TSI-3028HighImporting the DXA CMS-items into an existing BluePrint hierarchy which already has a configured Topology with discovery services and everything, is too hard. It can really only be done by importing too much and then rewiring the topology manually. We cannot assume that customers will figure this out. 
Update: it is really much easier now with the new verison of the master-only.zip! The scenario should be described in the documentation, but it looks good now.

2 - 

TSI-3014

TSI-3032

Medium

WORKS NOW! In all scenarios the model service is able to resolve links. The behavior is highly configurable now, since the removal of namespaces (including the xlink:href with the tcmuri in it) and the link resolving itself are controlled by separate properties.


Links are resolved in the model service. This makes it impossible for customers to write their own linking logic, which many of them have done. Please make this a configurable feature and default to false.

DXA publishes rich text like this:

<p>Hello <a href="tcm:90-123">world</a></p>

It resolves this into:

<p>Hello <a href="/world">world</a></p>

Although this is great, DD4T has link resolvers in the application which expect xlink:href attributes. The rich text should therefore be resolved into this:

<p>Hello <a href="/world" xlink:href="tcm:90-123" xmlns:xlink="http://www.w3.org/1999/xlink">world</a></p>


NOTE: latest version allows you to chose whether or not to resolve links AND remove namespaces. SDL is still discussing internally whether to split these to settings into:

  1. Resolve links (true or false, if true adds href="URL" without a namespace)
  2. Remove XMLNS (true or false, if true removes all namespace declarations and all xlink:* attributes)



9 - TSI-3033High

WORKS! See also issue 2.

The model service resolves links in rich text as well as the LinkUrl property in the DXA model, into hyperlinks without an extension. I understand why, because it is clean and many customers want it that way. But there are customers who currently route only **.html to their DD4T page controller. For these customers, the links will not work.


Could you make this behavior configurable? E.g.:

dxa.defaults.converter-extensionless-urls=true|false

It can default to true, so the default behavior remains the same as in the CTP.