Getting all items in all folders using the SharePoint Lists.asmx Web Service

One of the oddest things i’ve seen recently was the sudden inability to get all SharePoint list items in all folders on an old WSS 2.0 SharePoint Site Collection.
One of my smaller SharePoint site collections contains around 2000 items, including Folder items. In order to prepare for a manual migration to MOSS 2007, i am having to inventory all items and properly move them to their correct location in the new Microsoft Office SharePoint Server environment.
My initial attempt to get all items in all these lists was to use the standard “GetListItems” method of the Lists.asmx web service, but for some reason not all items were being returned.

For example, the “Personal” document library contained 25 Word documents but when queried, only 16 were being returned. I had to find the problem…which i did.
My code connects to my SharePoint Site Collection and gets a collection of all Document Libraries (ServerTemplate=”101″) only since this is what i was most interested in migrating. After i got a collection of document libraries, my code goes thru each library and gives me the most basic info about each document, including:

• File Directory
• Link to the File
• Who Created the File
• Who last modified it
• When was it last modified
• The File Size
• Full URL to the file

01.private void GetAllListItemsInThisList()

02.{

03.    XmlNode ndListItems = null;

04.  

05.    WSS2_LISTS.Lists service = new WSS2_LISTS.Lists();

06.    service.Url = @tbSiteURL.Text + @"/_vti_bin/lists.asmx";

07.    service.Credentials = new System.Net.NetworkCredential(@tbUserName.Text, @tbPassword.Text, @tbDomain.Text);

08.  

09.    XmlDocument xDoc = new XmlDocument();

10.    XmlNode ndQuery = xDoc.CreateNode(XmlNodeType.Element, "Query", "");

11.    XmlNode ndViewFields = xDoc.CreateNode(XmlNodeType.Element, "ViewFields", "");

12.    XmlNode ndQueryOptions = xDoc.CreateNode(XmlNodeType.Element, "QueryOptions", "");

13.  

14.    ndQuery.InnerXml = "";

15.    ndViewFields.InnerXml = "";

16.    ndQueryOptions.InnerXml = "<IncludeMandatoryColumns>FALSE</IncludeMandatoryColumns>";

17.  

18.    try

19.    {

20.        ndListItems = service.GetListItems(

21.                            "myListGUID",

22.                            "",

23.                            ndQuery,

24.                            ndViewFields,

25.                            null,

26.                            ndQueryOptions

27.            );

28.    }

29.    catch (Exception ex)

30.    {

31.        tbDetails.Text += ex.ToString();

32.    }

33.    if (ndListItems != null)

34.    {

35.        foreach (XmlNode node in ndListItems.ChildNodes)

36.        {

37.            if (node.Name == "rs:data")

38.            {

39.                for (int i = 0; i < node.ChildNodes.Count; i++)

40.                {

41.                    if (node.ChildNodes[i].Name == "z:row")

42.                    {

43.                        foreach (XmlAttribute attr in node.ChildNodes[i].Attributes)

44.                        {

45.                            if (attr.Name == "ows_FileDirRef")

46.                                tbDetails.Text += attr.Value.ToString().Substring(attr.Value.ToString().IndexOf("/") + 1) + "t";

47.                            if (attr.Name == "ows_LinkFilename")

48.                                tbDetails.Text += attr.Value + "t";

49.                            if (attr.Name == "ows_Author")

50.                                tbDetails.Text += attr.Value.ToString().Substring(attr.Value.ToString().IndexOf("#") + 1) + "t";

51.                            if (attr.Name == "ows_Modified")

52.                                tbDetails.Text += attr.Value + "t";

53.                            if (attr.Name == "ows_Modified_x0020_By")

54.                                tbDetails.Text += attr.Value + "t";

55.                            if (attr.Name == "ows_FileSizeDisplay")

56.                                tbDetails.Text += attr.Value + "t";

57.                            if (attr.Name == "ows_EncodedAbsUrl")

58.                                tbDetails.Text += attr.Value + "t";

59.                        }

60.                        tbDetails.Text += "rn";

61.                    }

62.                }

63.            }

64.        }

65.    }

66.}

Backup, Archive & Restore Office 365 SharePoint. FREE TRIAL

Running this code should have given me all documents (and it’s properies listed above) within each Document Library on this site collection….in theory, however, I was only getting 245 documents out of 2000+.
My first thought was that i was not recursively looking thru each folder within each document library, so i updated my “ndQueryOptions” to include a recursive query on this list:

 

1.ndQueryOptions.InnerXml = "<ViewAttributes Scope='RecursiveAll' IncludeRootFolder='True' />";

Thinking this would solve my problem, i quickly found out it wasn’t pulling back all documents as I had expected.
Oddly enough, when i queried my largest list (1400+ items, including folders) directly by specifying the list ID in the “GetListItems” method, i was getting only 100 documents! It didn’t really register with my right away until i noticed other lists with more than 100 documents were returning 100 as well.

Even though i have never run into this before, i discovered that the “Row Limit” parameter in the “GetListItems” method is actually there for a reason. You would think by specifying “null” as most people suggest means you don’t want a Row Limit returned, but that’s not the case. I had to specify a Row Limit of “100000″ (or something far greator than my library actually contained) to tell the “GetListItems” method to return everything in that list.

SharePoint Project Management Free Trial Extend SharePoint for Total Work Management

 

1.ndListItems = service.GetListItems(

2.                    "myListGUID",

3.                    "",

4.                    ndQuery,

5.                    ndViewFields,

6.                    "100000",

7.                    ndQueryOptions

8.    );

Setting this parameter along with the Recursive QueryOptions mentioned above, I was able to get back all documents in all folders within all document libraries on my site collection.
Finally, if you want to return ONLY the documents within a SharePoint document library and exclude the folder items, you can add this CAML to the “ndQuery” element:

 

1.ndQuery.InnerXml = "<Where><Eq><FieldRef Name='FSObjType' /><Value Type='Lookup'>0</Value></Eq></Where>";

The “FSObjType” (File System Object Type) for a folder is “1″, whereas a file is “0″. Specifying a “1″ will return all documents, excluding folder items.