Several people have found the bug that existed in setting a custom access denied page (or other custom layouts pages) in SharePoint 2013. The PowerShell cmdlet Set-SPCustomLayoutsPage “should” set the new custom page.
However, it never worked:
http://blog.randomdust.com/index.php/2013/07/custom-access-denied-page-in-sharepoint-2013/
http://deepinsharepoint.wordpress.com/2014/05/09/fix-custom-access-denied-page-in-sharepoint-2013/
Good news is, this has been fixed in April 2014 Cumulative Update(CU). We did find though, that there is one key condition for this to work properly. Your custom page MUST be in the Layouts directory. If you try to play the custom .aspx page in any other directory and then run the commands, setting the custom page will not work.
As you can see in the remarks on the recently updated page in MSDN here: http://msdn.microsoft.com/en-us/library/microsoft.sharepoint.administration.spwebapplication.updatemappedpage.aspx “the URL has to start with ‘/_layouts/'”.
Before you rush out to install this CU, there are a couple of regressions that exist in this CU that you should be aware of. Rather than rehash them all here, Todd Klindt has a list of theme here – http://www.toddklindt.com/blog/Regressions/SP2013Apr2014CU.aspx
