Releases

Release 2025.05.00

Featured

Power Fx

Power Fx is a low-code, open-source programming language developed by Microsoft. Built on Excel’s formula syntax, Power Fx combines familiarity with advanced capabilities for working with data.

We’ve now introduced Power Fx support in the Rule Builder. Instead of selecting a Property or Vocabulary, you can now select a Formula. This enables you to run Power Fx formulas directly within CluedIn and perform queries such as:

If(GetVocabularyKeyValue(Entity, "user.firstName") & " " & GetVocabularyKeyValue(Entity, "user.lastName") = GetVocabularyKeyValue(Entity, "user.fullName"), true, false)

or

CountRows(Entity.IncomingEdges) = 1

You can also execute Power Fx within rule actions by selecting the Formula Action. Please refer to the documentation for further details.

Auditlog

We’ve expanded the audit log coverage across most of the system to ensure all relevant changes and actions are tracked. You’ll now see significantly more detail within audit log tabs, which are also available in more areas throughout the platform.

Stream Logs, Entity Stream Logs and Export Target Logs

This is the first iteration of the Stream Log. Each stream now exposes basic information, including details about golden records that have been streamed out, export target-specific logs, and any health check status changes triggered during the streaming process. Basic filters are available, allowing you to filter by area and severity.

A new Streams tab has been added to Golden Records, displaying the latest details for each stream the golden record has been streamed through.

We’ve also introduced a Health Check tab on Export Targets, providing a historical record of all health checks for better visibility and traceability.

Additionally, there is a new setting (Retention Period in Days) under Stream Log settings. This allows you to control how long the logs mentioned above are retained.

Improved Data source list

We’ve introduced a streamlined design to help you more easily navigate your data sources. You can now filter by data source type and more easily drill down to the data you need.

Datasource Approval

You can now send items for approval as they are ingested into a data set. Approvals can be configured through property rules, pre-process rules, or via advanced mapping.

Important: If approval rules are set, only owners or users with the appropriate accountable claim can process the data directly. Otherwise, data will follow your configured approval flow.

Datasource Validation

You can now add validation to your data sets when in edit mode. This new feature helps you validate the data flowing into CluedIn, highlight issues, and improve data quality more easily.

Terminology Changes

To better align with industry standards and commonly used terminology, we’ve updated several key terms in CluedIn:

• Entity Type is now Business Domain
• Origin Entity Code is now Primary Identifier
• Codes/Entity Codes are now Identifier(s)

Vocabulary Key Picker

A new vocabulary key picker is now available wherever you select multiple vocabulary keys, such as in Streams, Clean, and Search. This enhancement provides a more intuitive workflow for finding and managing vocabulary keys, making it easier to see and control your selections.

Improved Search Bar

We’ve added a new search bar that allows users to:

• Quickly access suggested searches
• View recently searched items
• Use saved searches
• Filter results by business domain

Release Notes

CluedIn

Features

Rule Builder & Rule Actions

• Actions such as tagging can now be viewed in the rule data preview (#34199)
• PowerFx expressions now supported in Rule Builder (#35537)
• New Rule Actions:
  • Url Unescape Value
  • Standardize Phone Number
  • Round Up
  • Round Down
  • Round
  • Reverse
  • HTML Escape Value
  • Url Escape Value
  • Add Edge
  • Find Closest Match
  • To Partial Mask
  • Download and Set Image
• New rule operators allow finding valid and invalid lookup values (#45667)
• Basic throttling applied to OpenAI rule actions. There is a new setting in the Azure AI Services configuration page that lets you control the maximum number of requests (#47760)

Data Sources & Manual Data Entry Projects

• Ownership record now created when a new Manual Data Entry Project is initialized (#44344)
• Owners are automatically assigned when creating data sources (#42503)

Streams

• Improved stability of streams when working with large datasets (#40478)
• Non-owners/accountables now require approval to change stream states (start/pause/stop) (#40014)
• Notification shown if the platform fails to configure a stream in the background (e.g., starting or updating stream container) (#38949)
• Stream and connector health check logging implemented (#44574)

Entity Search & Shadow Data

• Robust tracking introduced for shadow data parts to improve shadow entity detection (#43127)
• Outgoing edge property values are now searchable via normal entity search through IndexedText (#41506)

Enabled via configuration:

Key	Default Value
ClueProcessing.IndexedText.EdgeProperties.Enabled	true

Deduplication & Clean

• Clean and Deduplication projects can now be filtered by state (#38256)
• New deduplication matching function handles non-equal value scenarios (#47268)
• Deduplication job made interruptible (#45586)

Access Control & Security

• Masking supported in Access Control (#43915)
• Default RACI behavior uses tenant management claim when no suitable users are found (#43314)

Vocabularies

• Deleting a vocabulary key or vocabulary now goes through the approval process (#44238)
• Vocabulary AllKeys endpoints can now exclude lookup keys (#46307)

Settings & Configuration

• Dedicated settings pages introduced for Azure services (#43922)
• Updated SQL Client library introduced breaking change for misconfigured certificates (#44188)

Workflow

• “Create” approvals will only trigger if the associated workflow is created and enabled (#48135)

Messaging

• RabbitMQ messaging made more robust with delivery confirmations (#40371)

Fixes

Core Platform & Stability

• ISO8601 dates stored in properties were incorrectly parsed by JsonSerializer.Deserialize due to Newtonsoft issue (#43394)
• Microsoft Extensions MemoryCache could cause task pool starvation in some cases, leading to unstable processing pods (#44559)
• Application could occasionally freeze when obtaining a lock (#40081)
• Processing Task Scheduler could mistakenly skip commands that should be executed (#47445)
• Entity history could display incorrect sort data (#45824)
• Entity version history sometimes displayed wrong edge properties when multiple edges pointed to the same reference point (#47707)
• Default timeout values for RabbitMQ and Redis connections improved for busy environments (#44654)

Added configuration keys:

Configuration Key	Default Value
Server.MessageBus.RequestedHeartbeatInSeconds	60
Server.MessageBus.TimeoutInSeconds	60
Server.CacheStore.ConnectTimeoutInSeconds	20
Server.CacheStore.SyncTimeoutInSeconds	20
Server.CacheStore.KeepAliveInSeconds	30

Deduplication & Clean

• Deduplication could fail under certain vocabulary typing configurations (#40688)
• DeduplicateRelationships not correctly handling edge properties during merging (#40354)
• Deduplication conflict page showing vocabulary key name twice (#13954)
• Vocabulary keys were not automatically created for new Clean columns (#30918)

Entity Merging

• Edge export behavior improved when export is disabled (#42186)
• Prevented shadow data names from overriding source system records (#45002)
• Improved handling of overlapping entity codes across large entity sets (#45396)
• Ensure that edges are processed correctly when IMergeEdgesPostProcessor is registered (#46379)
• Potential entity ID conflicts when re-ingesting deleted data resolved (#43982)
• Shadow entity creation was skipping edges with same code but different properties (#40354)
• Entity edge merging previously ignored differing properties; this behavior can now be controlled using Feature.EntityEdgeCollection.LegacyEdgeMergingWithoutConsideringPropertiesEnabled (#40575)

Messaging & Communication

• Inlining of processing commands disabled when ClueProcessing.Subscribe.Enabled is false (#41533)

Rules & Approvals

• Rule filter operators support vocabulary keys with Text data type and Untyped storage type (#42082)
• Greater Than and Less Than rule operators now respect mapped destination key type (#42261)
• RACI approvals now replay requests using configured server URL (#42251)
• Create action approvals now respect accountable-level requirements
• Approval required to create entity type using legacy endpoint (#43312)
• Ensure removed tags in glossary terms are logged during RACI approval (#38363)
• Ensure user and role ownership removal approval requests display the relevant names (#39270)

Audit Logs & Access Controls

• Actions on streams and access control policies no longer logged if unchanged (#42369)
• Rule access restricted to owning organization (#47226)
• Configurations restricted to owning organization (#47225)

Glossary & Vocabulary

• Glossaries used in archived Clean Projects can not be deleted (#41697)

Entity Handling & Display

• Fixed issues saving entities with extremely long property values (#43429)
• Corrected naming label from “No Title” to “No Name” for unnamed entities (#20935)
• Incorrect security controls on entity type counts (#36684)
• Special characters in entity codes no longer cause incorrect processing (#45178)
• Obsolete Insight and Subscribed Searches REST endpoints to reduce DB load during login and dashboard navigation (#34140)
• Improved accuracy of change type in ingestion log (#18167)
• Applied security access rules to hierarchy project entity views (#36697)

CluedIn.MicroServices

Features

Integrations & Dashboards

• Integrations dashboard have been improved (#44701)
• Filtered sources list added to integrations dashboard (#44701)
• Tasks dashboard moved to the main menu
• Pending reviews section added to show sources requiring approval or quarantine review (#24456)

Data Handling & Storage

• Dataset clues now store dataSetId and receiptId (for endpoints only) (#44962)
• Original dataset file data can now be downloaded (#46119)

Datasets & Manual Entry

• Owners support added to dataset entities (#42342)
• Approval records added for manual data entry projects (#24456)
• Daily emails sent to owners of sources (#24456)
• Validation records functionality introduced in datasets (#44778)
• Profiling job added to measure code overlaps (anomalies) in the system
• Enabled dataset processing after modifying records (#43426)
• Supported operations on date fields in edit mode (#43597)
• Search functionality added to dataset preview (#46009)
• Operations and filters enabled for SQL and endpoint datasets (#42094)
• Applied proper filtering mechanisms within data sets (#47153)

Fixes

Crashes & Stability

• Prevented crash when uploading password-protected XLSX files
• Resolved issue when fetching approval stats for a non-existent index
• Addressed crash during property ingestion updates (#47354)
• Corrected daily report logic after modifying records (#43426)
• Improved edit mode stability in multiple scenarios (#43539, #43597)
• Ensured download and approval clue handling works as expected (#46689)
• Avoided duplicate clue submissions in edit mode (#45186)
• Improved reliability of manual approval clue handling
• Improved approval clue behavior during rejection (#45077)

Datasets & Data Handling

• Validated endpoint data length correctly (#46007)
• Addressed inconsistencies in ingestion property counts (#46007)
• Removed validation barriers when deleting extra fields (#47236)
• Aligned validation logic for age, count, and empty values (#47236)
• Updated validation totals and ensured refresh triggers correctly (#47256, #47258)
• Improved total value operation logic in ingestion (#47354)
• Cleaned up behavior for data source deletion (#47522)
• Resolved issues with autovalidation loops (#47254)
• Enabled validation caching (#47254)
• Ensured mapping properties are properly ignored when needed (#47272)
• Automatically reset validations when fields or mappings are adjusted (#48239)
• Defaulted supportsIngestionSourceId on dataset creation (#48028)
• Disabled profiling in specific scenarios and ensured endpoint ingestion reliability (#47294)
• Supported filtering data sets by validation status (#48143)
• Corrected boolean validation behavior (#48162)

Approvals & Permissions

• Ensured approval stats are visible to accountable users in dashboards (#43414)
• Restricted data source page access to assigned owners only (#45230)
• Blocked non-owners from deleting approval-required rules (#45123)
• Applied access checks based on providerDefinitionId (#46487)
• Handled submissions queries reliably (#46563)
• Corrected logic for rules requiring approval at the property level
• Prevented approval clue issues tied to index existence
• Enforced clearer behavior for rule creation approvals

Ingestion & Profiling

• Aligned date handling within ingestion profiling (#46084)
• Enhanced ingestion and validation performance (#47254)
• Prevented duplication in profiling jobs (#46564)
• Launched anomaly profiling startup jobs with notifications
• Triggered alerts for duplicated entity codes during anomaly checks (#45549)
• Displayed separated and paged source statistics (#45187)
• Managed consumers monitoring and re-registration on failure (#46175)
• Disabled repeated alerts when consumers are auto-recreated (#46175)
• Refined profiling intervals to prevent duplication (#46564)
• Cleaned logs for bridge mode ingestion operations (#47257)
• Ensured change types in ingestion logs are accurate
• Registered consumers correctly during pod startup
• Enabled automatic recovery for missing consumers (#46175)
• Scheduled recurring jobs to reset consumers if needed (#46175)
• Suppressed notifications for self-healing consumer restarts (#46175)
• Refined command execution logic in the processing task scheduler (#47445)

CluedIn.UI

Features

Streams & Monitoring

• Added backlog and exported statistics to the stream page for quick monitoring

Dataset Operations & Manual Data Entry

• Enabled operations & filters for SQL and endpoint datasets (#42094)
• Added support for dataset owners (#42342)
• Added search functionality in dataset preview
• Enabled removal of records from Elasticsearch by receiptId
• Preview of ingested records by receiptId added to the Data Set Ingestion Reports page (#46009)
• Disabled bridge mode switching if dataset is in edit mode and added fields are used in mapping
• Added ability to download original dataset file (#46119)
• Allowed more rule operators for lookup keys’ EntityCode composite key (#45444)
• Added ability to remove records in dataset ‘Edit Mode’ (#41815)
• Added validation records in datasets (#44778)
• Add a deduplication code API to retrieve codes that are duplicates (#45391)
• Add a menu item called “Ingestion Anomalies” under the monitoring of a data set in order to verify if there are issues in the current ingestions (#45548)
• Added approval records in manual data entry projects (#24456)
• Moved tasks dashboard to main menu with a Pending Reviews section showing sources needing approval/quarantine review (#24456)
• Access to datasource pages now granted to owners (#45230)
• Added select-all functionality in tables, activated in dataset preview (#45003)
• Download original file (#46584)

User Experience & Interface

• Adjusted “Engine Room > Configuration Groups” page to a tile layout with new icons (#38809)
• Updated terminology from “entity type(s)” to “business domain(s)” and “codes” to “identifiers” (#44623)
• Added step progress indicators to new home dashboard carousel modals (#45761)
• Updated search bar: selected business domains now appear in placeholder text (#45753)
• Changed new home dashboard sources KPI to show breakdown tooltips (#43608)
• Changing appearance of long user names in Audit Log (#41295)
• Changing appearance of long vocabulary keys in edit mapping (#41741)
• Changing Unmerge All button disabled style (#43293)
• Changing appearance of field with no value in View Changes panel (#39884)
• Changing View Changes panel name (#38286)
• Changing names in Audit Log changes panel (#39823)
• Changing appearance of Ingestion reports tab without permission (#42550)
• Changing appearance of user menu panel (#43372)
• Changing naming of places where Glossary Term is used (#42123)
• Changing appearance of tree view after creating term (#42495)
• Changing appearance of disable tooltip for toggle button in Export Targets (#41314)
• Changing Merge All panel stepper logic (#40265)
• Changing place where role added to owner opens (#43198)
• Added button in the Enricher to remove data parts

Audit & Access

• Introduced new Audit Log Action type: “Action” (#41811)
• Added audit logs tab to Hierarchy, Settings, User Management, and Roles pages (#44082)
• Support added for masking in Access Control (#43915)
• Changing configuration preview status field (#16965)
• Changing incorrect breadcrumbs logic (#38107)

Navigation & Dashboard

• Redeveloped Integrations dashboard (#44701)
• Introduced new home page dashboard layout
• Introduced new search bar behind a feature flag (#42554)
• Search icon button added to the new search bar (#45506)
• Adjusted search suggestions for home and “My Tasks” sections (#45638)
• Improved datasources list layout. Added panel to see datasources by group. Improved empty states (#44701)

Ownership & Notifications

• Added notifications for source owners when required (#42342)

Vocabulary & Form Enhancements

• Introduced new vocabulary key transfer list with extended filtering and search capabilities (#36524)

AI & Copilot

• Added Copilot prompt suggestions and sample prompts (#38317)
• Introduced AI Configurations module (#45438)

Matching & Rules

• Feature flag added for “Not equals” matcher in deduplication matching rules (#47297)
• PowerFx expression support added in Rule Builder for both filters and actions (#35541)

Fixes

General UI/UX Changes

• Changing appearance of long edge property key in Relations List (#38543)
• Changing appearance of pencil button in Edit Mapping (#39416)
• Changing on-hover appearance for items in Global Search (#39735)
• Adjusted ‘Create’ sidemenu button margin (#42453)
• Truncate issue with “Changed By” and “Requested By” columns in Audit Log and Pending Changes List (#42202)
• Copilot settings panel auto-closing on settings submit error (#41171)
• Copilot example prompts not rendering correctly (#46424)
• Incorrect breadcrumb text in the Integrations page (#47763)
• Group name rendering
• Fixed collapsing panel when doing navigation
• Fixed non-disabled element on approval tab
• Show pending approval badge on manual project (#45299)

Search Bar Enhancements

• Truncating long values in the search bar (#44803)
• Hiding search results and module options based on feature flags (#44769)
• Tooltip for long saved search names on hover (#45117)
• Saved search panel open with titled text (#44750)
• Show 7 saved searches in dropdown; consistent visibility on focus (#45231)
• Adjusted dropdown height and tooltip count fitting (#45216)
• Search bar dropdown matches width of search bar (#45114)
• Business domain dropdown updates: show “View all…”, heading/tooltips adjusted (#44841, #44907)
• Prevent dropdown if fewer than 3 characters entered (#44984)
• Suggested items not working correctly (#47298)
• Reduced excessive queries (#47118)
• Fixed tooltip truncation and terminology issues (#44992)
• Removed forced lowercase from search URL scope (#45262)
• Incorrect query used for source count (#47241)
• Filtering in golden record list with no associated records (#44886)
• Business domain filter not working with saved searches applied (#47029, #47783)

Mapping & Rule Builder

• Issue when ‘Reset mapping’ was unavailable after removing golden records (#47486)
• Error when viewing change requests in rule builder (#46418)
• Reject button availability changes in Deduplication > Matches (#35312)
• Reset mapping after records removed (#46225)
• Enricher name not updated after saving (#45941)
• Access Control Policy Action now resets action parameters (#46382)

Copilot & AI

• Issue with Copilot crashing due to configuration errors (#46424)
• Incorrect behaviour saving Copilot settings (#41617)

Home Dashboard

• KPI counts not wrapping; visual fixes (#45249)
• Business domain chart title and tooltip text changed (#45064)
• Adjusted source count to reflect manual data and crawlers (#43420, #46425)
• Count tiles updated (#40522)
• Chart tooltip info icon added (#41071)

Datasources & Integrations

• Datasource list layout improvements (#44701)
• Sources page access for owners (#45230)
• Crawlers navigation added (#45072)
• Separate sources stats tables (#45187)
• Issue for approval and duplicate checks (#43516)
• Get 20 latest projects to show stats (#45187)
• Renamed “Integrations” to “Crawlers”, added tile, fixed approval bugs (#45072)

Saved Searches

• Badge removed after deletion of active saved search (#44744)
• Saved searches dropdown spacing fixes (#45181)
• Adjusted saved search queries polling and consistency (#44985)

Audit & Ownership

• Audit log for Clean filtered by object ID (#43942)
• Paging on ownership (#45062)

Performance & Stability

• Search box performance on dataset preview (#46009)
• Highlighting and selection issues on dataset preview (#47183, #45958)
• Manual Lookup edit special character crash fix (#46320)
• Incorrect terminology in multiple modules (streams, ingestion, history, explain log) (#44991, #44993, #44994)
• Cleaning data fixes (#47124)
• Changing terminology in Action Center and rule filters (#44999, #44991)

Runtime-Environment

Features

• Added field to determine if dataset is in edit mode (#42094)
• Added DescriptionId to AuditLog.ChangeSet (#41826)
• Added support for datasource owners (#42342)
• Added approval records on manual data entry project (#24456)
• Added validation records ability in datasets (#44778)
• Added table to support new landing zones (#44865)
• Added ability to store dataSetId and receiptId in clues (#44962)
• Added table to monitor basic metrics for ingestions (datasets and manual data entry project) (#45227)
• Added a UIType column to the auditLogChanges table (#45415)
• Added ProfilingAnomalyMetric table for tracking ingestion anomalies
• Added ProfilingDuplicateCodeFactMetric table for tracking duplicate codes (anomalies)
• Implemented Connector Health Status Log
• Corrected AccessControlOwnershipProvider ObjectTypeIds (#45188)
• Ensured admin.role and admin.entitytypes claims have minimum level of “Informed” (#39738)
• Created DataSetValidationOperations table (#44820)
• Changed columns in ProfilingAnomalyMetric table
• Re-introduced constraint to prevent duplicate hourly profiling reports in ingestion reports (#46564)
• Defaulted supportsIngestionSourceId to false to prevent older datasets from defaulting to true and avoid missing-data issues (#48028)

Packages

For this release, kindly utilize the precise versions listed below for the following packages

Connectors

Name	Version
CluedIn.Connector.AzureDataLake	4.5.0
CluedIn.Connector.AzureDedicatedSqlPool	4.0.2
CluedIn.Connector.AzureEventHub	4.5.0
CluedIn.Connector.AzureServiceBus	4.5.0
CluedIn.Connector.Http	4.5.0
CluedIn.Connector.SqlServer	4.5.0
CluedIn.PowerApps	4.5.0
CluedIn.Connector.Dataverse	4.5.0
CluedIn.Connector.OneLake	4.5.0
CluedIn.Connector.FabricOpenMirroring	4.5.0

Enrichers

Name	Version
CluedIn.ExternalSearch.Providers.DuckDuckGo.Provider	4.5.0
CluedIn.ExternalSearch.Providers.PermId.Provider	4.5.0
CluedIn.ExternalSearch.Providers.Web	4.4.1
CluedIn.Provider.ExternalSearch.Bregg	4.4.1
CluedIn.Provider.ExternalSearch.ClearBit	4.4.1
CluedIn.Provider.ExternalSearch.CompanyHouse	4.5.0
CluedIn.Provider.ExternalSearch.CVR	4.4.1
CluedIn.Provider.ExternalSearch.Gleif	4.4.1
CluedIn.Provider.ExternalSearch.GoogleMaps	4.5.0
CluedIn.Provider.ExternalSearch.KnowledgeGraph	4.5.0
CluedIn.Provider.ExternalSearch.Libpostal	4.4.1
CluedIn.Provider.ExternalSearch.OpenCorporates	4.5.0
CluedIn.Provider.ExternalSearch.Providers.VatLayer	4.5.0
CluedIn.Provider.ExternalSearch.Providers.AzureOpenAI	4.4.1
CluedIn.Provider.ExternalSearch.GoogleImages	4.4.1
CluedIn.Provider.ExternalSearch.RestApi	4.5.0

Crawlers

Name	Version
CluedIn.Crawling.MasterDataServices	4.5.0
CluedIn.Purview	4.5.0

Other

Name	Version
CluedIn.Vocabularies.CommonDataModel	4.5.0
CluedIn.EventHub	4.5.0
CluedIn.EnterpriseFlows.PowerAutomate	4.5.0

Tags

Controller

Docker Image	Tags
cluedin/controller	2025.05.00, 2025.05, 4.5, 4.5.0, 4.5.0_116093

Gql

Docker Image	Tags
cluedin/cluedin-ui-gql	2025.05.00, 2025.05, 4.5, 4.5.0, 4.5.0_116090

Microservices

Docker Image	Tags
cluedin/data-source	2025.05.00, 2025.05, 4.5, 4.5.0, 4.5.0_116092
cluedin/data-source-processing	2025.05.00, 2025.05, 4.5, 4.5.0, 4.5.0_116092
cluedin/data-source	2025.05.00, 2025.05, 4.5, 4.5.0, 4.5.0_116092
cluedin/data-source-processing	2025.05.00, 2025.05, 4.5, 4.5.0, 4.5.0_116092
cluedin/data-source-submitter	2025.05.00, 2025.05, 4.5, 4.5.0, 4.5.0_116092
cluedin/data-source	2025.05.00, 2025.05, 4.5, 4.5.0, 4.5.0_116092

Runtime

Docker Image	Tags
cluedin/neo4j	2025.05.00, 2025.05, 4.5, 4.5.0, 4.5.0_116094
cluedin/openrefine	2025.05.00, 2025.05, 4.5, 4.5.0, 4.5.0_116094

Server

Docker Image	Tags
cluedin/cluedin-server	2025.05.00, 2025.05, 4.5, 4.5.0, 4.5.0_116098, 4.5.0_116098-alpine, 4.5.0-alpine, 4.5-alpine
cluedin/cluedin-server	2025.05.00, 2025.05, 4.5.0_116098-ubuntu, 4.5.0-ubuntu, 4.5-ubuntu
cluedin/nuget-installer	2025.05.00, 2025.05, 4.5, 4.5.0, 4.5.0_116098, 4.5.0_116098-alpine, 4.5.0-alpine, 4.5-alpine
cluedin/nuget-installer	2025.05.00, 2025.05, 4.5.0_116098-ubuntu, 4.5.0-ubuntu, 4.5-ubuntu

Ui

Docker Image	Tags
cluedin/ui	2025.05.00, 2025.05, 4.5, 4.5.0, 4.5.0_116128