Figured it out. I switched everything to inner joins except the Note one and re-did the entire query. It is working now.

For anyone that cares, the new query looks like

Code:
SELECT     Client.Name, Company.Name AS [Company Name], Note.DateCol, UDF.Description
FROM         AMGR_Client_Tbl AS Client INNER JOIN
                      AMGR_Client_Tbl AS Company ON Client.Client_Id = Company.Client_Id INNER JOIN
                      AMGR_Notes_Tbl AS Note ON Client.Contact_Number = Note.Contact_Number AND Client.Client_Id =
                          (SELECT     TOP (1) Client_Id
                            FROM          AMGR_Notes_Tbl
                            WHERE      (Client_Id = Note.Client_Id) AND (Contact_Number = Note.Contact_Number) AND (Note_Type = 'Contact')) LEFT OUTER JOIN
                      AMGR_User_Field_Defs_Tbl AS UDF ON UDF.Type_Id = '2' AND UDF.Code_Id =
                          (SELECT     Code_Id
                            FROM          AMGR_User_Fields_Tbl
                            WHERE      (Type_Id = '2') AND (Client_Id = Client.Client_Id) AND (Contact_Number = Client.Contact_Number))
WHERE     (Client.Contact_Number <> 0) AND (Company.Contact_Number = 0) AND (Note.Note_Type = 'Contact')