Client Background

The client, a local organisation, relied on an on-premises server to store critical documents, metadata, and business records across multiple departments. As part of a modernisation initiative aligned with government cloud-first strategies, the organisation sought to transition to SharePoint Online (SPO) for improved accessibility, collaboration, and security.

Challenge

The legacy SharePoint 2016 system had become increasingly complex and expensive to maintain. Key issues included:

  • Rising security risks associated with aging infrastructure.

  • High maintenance costs for servers and backups.

  • Thousands of files, folders, and custom lists with embedded metadata and HTML content requiring precise migration.

  • Data integrity concerns include broken links, non-standard tags, and duplicate entries.

  • The need for business continuity, minimal downtime, and full auditability during and after migration.

Solution by Data Gravity

Service Provided: Data Migration & Decommissioning
Technologies Used: Python, SQL Server, SharePoint Online (SPO), ShareGate

Process Overview

  1. Content Extraction:

    • Extracted structured and unstructured data from SharePoint 2016 using ShareGate and SQL queries.

    • Identified and indexed custom metadata fields, HTML-based content, and file references for cleanup.

  2. Transformation & Staging:

    • Used Python scripts for:

      • Parsing and sanitizing HTML content embedded in SharePoint pages and columns.

      • Renaming files for consistency and compliance.

      • Cleaning and reformatting metadata to align with SharePoint Online standards.

    • Built an ETL pipeline to process, validate, and stage data in SQL Server before final upload.

  3. Migration & Validation:

    • Employed ShareGate for batch migration of documents, lists, and metadata.

    • Validated successful transfers and flagged broken links or missing files for manual review.

    • Ensured metadata fidelity, user permissions, and folder structures were preserved.

  4. Decommissioning & Governance:

    • Implemented a decommissioning strategy for the legacy system.

    • Logged missing or corrupted files and generated detailed audit reports for compliance.

    • Supported post-migration user access testing and documentation.

Results

  • Successfully migrated over 75,000 files and 120 custom lists, encompassing more than 300 metadata fields.

  • Achieved a 20% reduction in infrastructure costs by retiring on-premises servers and maintenance.

  • Improved access speed and document searchability across departments.

  • Strengthened compliance with the Australian Government’s cloud-first policy.

  • Delivered comprehensive migration reports and audit logs to satisfy governance and IT policy requirements.

Client Testimonial

“Data Gravity’s professionalism and technical capability ensured a smooth and secure transition to SharePoint Online. Their automation and attention to detail significantly reduced manual work and risk, enabling our teams to focus on delivery instead of downtime.” IT Operations Manager

Key Takeaways

  • Strategic Automation: Python and ShareGate enabled high-volume migration with minimal manual effort.

  • Data Integrity: Custom scripts and SQL staging ensured metadata, links, and content were preserved and validated.

  • Governance Ready: Full audit trails and decommissioning logs ensured transparency and regulatory compliance.

  • Long-Term Value: The migration future-proofed the organization’s content management infrastructure while improving user experience and security posture.