Could not normalize the SPItem after installing SharePoint Connector for FAST

I had setup SharePoint connector on my FAST server and provided SharePoint username and password which has go read access to sharepoint site. However, when starting the document processing the content retreiver fails and complains about not being able to retreive content because of unauthorized access to the item its trying to retrieve. Following is the exact error details: com.fastsearch.esp.connector.sharepoint2007.adapter.SPAdapterController: Could not normalize the SPItem 3b370549-3f6f-4d3f-b7c8-1d98884101ce;450526c0-5b6c-452a-a655-b1093b7e9d46;1. Exception: com.fastsearch.esp.cctk.framework.NormalizationException: Could not get content for the url: 'http://moss/Lists/Announcements/DispForm.aspx?ID=1'. Exception: com.fastsearch.esp.cctk.framework.AdapterException: Error downloading file: http://moss/Lists/Announcements/DispForm.aspx?ID=1 from server: moss.gnosispower.com HTTP Status code: 401. HTTP Status msg: Unauthorized Looking on the IIS logs for the get request for "'http://moss/Lists/Announcements/DispForm.aspx?ID=1" no username is associated. Not sure which user account document processing stage is using to retrieve the content.

So investigated further on this issue and ended up solving it by configuring Web Authentication Proxy. For some reason one of the dcoument processing stage when requesting content from sharepoint was not using the SharePoint username/password specified in the configuration. So have to setup Web Authentication Proxy to force it to use a specified account while retrieving content.

Hope it might help some one, sometime :-)