Development work to prep AP, snowstorm, release systems, etc

• MAINT-2493 - Annotations: Change annotations refset column languageCode to languageDialectCode (this relates to content stored in the Terminology Server, rather than the column headings in the file) - complete
• INFRA-12849 - Annotations: Change annotations refset column languageCode to languageDialectCode - complete
• INFRA-12258 - Add attribution to Orphanet - in progress
• INFRA-12737 and INFRA-12763 these relate to the data type metadata we declare for these two refsets in the Refset Descriptor - now complete.

PWI will be starting to add attributions to Production w/b 13 May 2024 and we also need to test out a release that pulls refset descriptor changes from Snowstorm rather than an externally maintained S3 folder.

Changes to the Annotations Refset format

:

• The languageCode field was specified as the two characters of ISO-639-1 code for the language of the annotation text.
  • The change is to include support for specifying dialect when it is applicable by adhering to RFC 5646.
  • This allows the combination of two characters of ISO 639-1 code and two uppercase letters of country code (ISO 3166) separated by a hyphen.
• The "languageCode" will be changed to "languageDialectCode" in the field column of BOTH REFSETS. (see

  Jira
  server IHTSDO JIRA serverId b202d822-d767-33be-b234-fec5accd5d8c key INFRA-12849

  )
  • The metadata concept 1304275002 |Language or dialect code (foundation metadata concept)| will be used for the metadata of this column in the descriptor refset.
• The changes will apply to both the Member Annotation String Value reference set + Component Annotation String Value reference set
  • CONFIRMED IN VERSION 2 OF THE JUNE 2024 PreProd RELEASE:
    • Component Annotations (1292992004)
      • < id    effectiveTime    active    moduleId    refsetId    referencedComponentId    languageCode    typeId    value
        ---
        > id    effectiveTime    active    moduleId    refsetId    referencedComponentId    languageDialectCode    typeId    value
    • Member Annotations (1292995002)
      • < id    effectiveTime    active    moduleId    refsetId    referencedComponentId    referencedMemberId    languageCode    typeId    value
        ---
        > id    effectiveTime    active    moduleId    refsetId    referencedComponentId    referencedMemberId    languageDialectCode    typeId    value

NOTE:

1. EDITIONS & EXTENSIONS REQUIRE SAME CHANGES AS INT EDITION (above)

Fixes to the refsetDescriptor records

(see INFRA-12737 for details):

• 1. The AttributeType for the (attributeOrder=0) refsetDescriptor record for both Member + Component Annotations needs to be changed from:
  • 900000000000461009 |Concept type component|
    • to
  • 900000000000460005 |Component type (foundation metadata concept)|
  • (see comments in 5.2.4.9 Member Annotation String Value Reference Set + https://jira.ihtsdotools.org/browse/INFRA-12763 for details)
    • CONFIRMED IN VERSION 2 OF THE JUNE 2024 PreProd RELEASE:
      • < 6cc1d4ca-32fd-44cb-8ad3-e49f23ec7760    20231201    1    900000000000012004    900000000000456007    1292992004    900000000000518009    900000000000461009    0
        ---
        > 6cc1d4ca-32fd-44cb-8ad3-e49f23ec7760    20240601    1    900000000000012004    900000000000456007    1292992004    900000000000518009    900000000000460005    0
        
      • < f6d9647b-6ee6-400d-9c7f-77818fa1f986    20231201    1    900000000000012004    900000000000456007    1292995002    900000000000518009    900000000000461009    0
        ---
        > f6d9647b-6ee6-400d-9c7f-77818fa1f986    20240601    1    900000000000012004    900000000000456007    1292995002    900000000000518009    900000000000460005    0
        
    • Need cross check from Yong/Peter/Maria...CONFIRMED VIA EMAIL AT 11:19 on 14/05/2024
    • Need verification in final Prod release for June 2024...CONFIRMED 15/05/2024 - International Edition Traceability matrix (June 2024)
• .
• 2. The languageDialect column for a) the (attributeOrder=2) refsetDescriptor record for Member Annotations + b) the (attributeOrder=1) refsetDescriptor record for Component Annotations
• , needs to be changed from:
  • 1296895003 |Language code (foundation metadata concept)|
    • to the new concept
  • 1304275002 |Language or dialect code (foundation metadata concept)|
  • (see comments in https://jira.ihtsdotools.org/browse/MAINT-2493)
  • .
  • SORRY Peter G. Williams - the refsetDescriptor records in MAIN have been updated as per items 1 & 3 (above and below), but haven't had the Language Code value changed to the new one
    • - which should now be 1304275002 as specified by Yong above:

• • • 2c0d0378-c28c-4705-b8af-f5f0f8276dfd    20231201    1    900000000000012004    900000000000456007    1292992004    1296895003    900000000000465000    1
      
    • afe9824b-1323-4407-a0b7-a028ae2b780a    20231201    1    900000000000012004    900000000000456007    1292995002    1296895003    900000000000465000    2
      
  • .
  • Confirmed now working in Version 1 of the May 2024 Prod INT Edition release - International Edition Traceability matrix (June 2024)
  • .
• 3. The AttributeType for a) the (attributeOrder=4) refsetDescriptor record for Member Annotations (1292995002) + b) the (attributeOrder=3) refsetDescriptor record for Component Annotations (1292992004), needs to be changed from:
  • 900000000000459000 (AttributeType)
    • to
  • 900000000000465000 (String)
  • (see comments in https://ihtsdo.freshdesk.com/a/tickets/49483)
  • They are correct in the following Docs, but the entries in the descriptor records are incorrect and need correcting.  
    • https://confluence.ihtsdotools.org/display/WIPRELFMT/5.2.4.9+Member+Annotations+String+Value+Reference+Set + 
    • https://confluence.ihtsdotools.org/display/WIPRELFMT/5.2.4.8+Component+Annotations+String+Value+Reference+Set
    • CONFIRMED IN VERSION 2 OF THE JUNE 2024 PreProd RELEASE:
      • < 54af9962-2a35-4538-86f3-a8c38de9200b    20231201    1    900000000000012004    900000000000456007    1292995002    900000000000519001    900000000000459000    4
        ---
        > 54af9962-2a35-4538-86f3-a8c38de9200b    20240601    1    900000000000012004    900000000000456007    1292995002    900000000000519001    900000000000465000    4
      • < b46405ec-5ff1-46a3-a9cc-b2ef2bba23cc    20231201    1    900000000000012004    900000000000456007    1292992004    900000000000519001    900000000000459000    3
        ---
        > b46405ec-5ff1-46a3-a9cc-b2ef2bba23cc    20240601    1    900000000000012004    900000000000456007    1292992004    900000000000519001    900000000000465000    3
        
    • Need cross check from Yong/Peter/Maria...CONFIRMED VIA EMAIL AT 11:19 on 14/05/2024
    • Need verification in final Prod release for June 2024...CONFIRMED 15/05/2024 - International Edition Traceability matrix (June 2024)

NOTE:

1. EDITIONS REQUIRE SAME CHANGES AS INT EDITION (above)
2. EXTENSIONS REQUIRE NO CHANGES, because as per the spec (5.2.4.1 Reference Set Descriptor):

1. Creation of a Reference set descriptor is optional when creating a reference set in another Extension. If a descriptor is not created, the descriptor of the closest ancestor of the reference set is used when validating reference set member records.

2. This is the same approach as all other International refsets that are also included in the Extensions - MRCM Refsets, SimpleRefset, SimpleMapRefset, etc.  They are NOT included in the Extensions RefsetDescriptor files, and are included in the Edition RefsetDescriptor files only because they are part of the International module and are therefore included as part of the International content that comprises the Edition packages.