Well I found a way to make it work (and keep my readonly property)

The solution is to keep our private variable, keep our read-only property and Add a read-write property with the option :
<System.ComponentModel.EditorBrowsable(System.ComponentModel.EditorBrowsableState.Never)>

So when deserialize the object, the system will be able to edit the private data using the new read-write property
and the new read-write property is invisible from the Intelisense (using the option), so it keeps our way of having a read-only property.

It is not perfect, but it works.

Code:
Private _ID as string

Public ReadOnly Property ID As String
    Get
          Return Me._ID
    End Get
End Property


<System.ComponentModel.EditorBrowsable(System.ComponentModel.EditorBrowsableState.Never)>
Public Property _ID_edit As String
     Get
          Return Me._ID
     End Get
     Set(value As Boolean)
          Me._ID = value
     End Set
End Property