standardized-messages
Table of Contents
- Overview
- Design Goals
- Core Schema
- Platform-Specific Mappings
- Database Schema
- API Design
- Implementation Notes
- Migration Strategy
- Testing Strategy
- References
Overview
This document defines a platform-agnostic message schema for centralizing messaging data from multiple platforms (Slack, Microsoft Teams, Discord, etc.). The schema normalizes platform-specific structures into a unified format while preserving platform-specific metadata for bi-directional synchronization.
Design Goals
- Platform Agnostic: Work with any messaging platform
- Extensible: Easy to add new platforms and message types
- Bi-directional: Support both read and write operations
- Lossless: Preserve platform-specific data for accurate sync
- Type Safe: Well-defined TypeScript interfaces
Core Schema
Base Message
The foundation for all messages across platforms:
interface StandardMessage { // Universal Identifiers id: string; // Internal unique ID (UUID) platformId: string; // Original platform message ID platform: MessagePlatform; // Source platform
// Content content: MessageContent;
// Author author: StandardUser;
// Context conversation: ConversationContext; thread?: ThreadContext; // Optional threading
// Timestamps timestamp: Date; // Message creation time editedTimestamp?: Date; // Last edit time deletedTimestamp?: Date; // Deletion time (soft delete)
// Interactions reactions: Reaction[]; mentions: Mention[];
// Metadata metadata: MessageMetadata; platformSpecific: Record<string, any>; // Platform-specific data}
enum MessagePlatform { SLACK = 'slack', TEAMS = 'teams', DISCORD = 'discord', TELEGRAM = 'telegram', WHATSAPP = 'whatsapp', INTERNAL = 'internal'}Message Content
interface MessageContent { type: MessageType; text?: string; // Plain text content html?: string; // Rich text (HTML) markdown?: string; // Markdown formatted blocks?: ContentBlock[]; // Structured content blocks attachments: Attachment[];}
enum MessageType { TEXT = 'text', FILE = 'file', IMAGE = 'image', VIDEO = 'video', AUDIO = 'audio', SYSTEM = 'system', // System notifications CALL = 'call', // Call events POLL = 'poll', SNIPPET = 'snippet', // Code snippets LINK_SHARE = 'link_share'}
interface ContentBlock { type: 'text' | 'code' | 'quote' | 'list' | 'heading'; content: string; language?: string; // For code blocks level?: number; // For headings/lists}User Schema
interface StandardUser { id: string; // Internal unique ID platformId: string; // Platform-specific user ID platform: MessagePlatform;
// Identity username: string; displayName: string; email?: string;
// Profile avatar?: string; // Avatar URL status?: UserStatus; timezone?: string;
// Metadata isBot: boolean; isGuest: boolean; roles: string[]; // Platform roles
platformSpecific: Record<string, any>;}
interface UserStatus { state: 'active' | 'away' | 'dnd' | 'offline'; message?: string; emoji?: string; expiresAt?: Date;}Conversation Context
interface ConversationContext { id: string; // Internal conversation ID platformId: string; // Platform conversation/channel ID platform: MessagePlatform;
type: ConversationType; name?: string; // Channel/group name topic?: string;
participants: StandardUser[];
metadata: { isPrivate: boolean; isArchived: boolean; createdAt: Date; createdBy: StandardUser; };
platformSpecific: Record<string, any>;}
enum ConversationType { DIRECT = 'direct', // 1:1 DM GROUP = 'group', // Group chat CHANNEL = 'channel', // Public channel PRIVATE_CHANNEL = 'private_channel', THREAD = 'thread' // Thread within conversation}Thread Context
interface ThreadContext { id: string; platformId: string; parentMessageId: string; // ID of parent message rootMessageId: string; // ID of thread root replyCount: number; participantCount: number; lastReplyAt?: Date;}Attachments
interface Attachment { id: string; type: AttachmentType;
// File info filename?: string; mimeType?: string; size?: number; // Bytes
// URLs url: string; // Download/view URL thumbnailUrl?: string; previewUrl?: string;
// Media metadata dimensions?: { width: number; height: number; }; duration?: number; // Seconds (for audio/video)
// Link preview title?: string; description?: string;
platformSpecific: Record<string, any>;}
enum AttachmentType { FILE = 'file', IMAGE = 'image', VIDEO = 'video', AUDIO = 'audio', LINK = 'link', DOCUMENT = 'document'}Reactions
interface Reaction { id: string; emoji: string; // Unicode or platform-specific emojiId?: string; // Custom emoji ID users: StandardUser[]; // Users who reacted count: number; createdAt: Date;}Mentions
interface Mention { type: MentionType; userId?: string; // For user mentions text: string; // Display text position: { start: number; end: number; };}
enum MentionType { USER = 'user', CHANNEL = 'channel', EVERYONE = 'everyone', HERE = 'here', ROLE = 'role'}Message Metadata
interface MessageMetadata { source: 'api' | 'webhook' | 'bot' | 'import'; version: number; // Schema version
// Sync state syncStatus: SyncStatus; lastSyncedAt?: Date; syncErrors?: SyncError[];
// Flags isPinned: boolean; isStarred: boolean; isImportant: boolean; isEdited: boolean; isDeleted: boolean;
// Search/indexing searchableText: string; // Normalized for search tags: string[];
// Custom data custom: Record<string, any>;}
interface SyncStatus { synced: boolean; direction: 'inbound' | 'outbound' | 'bidirectional'; lastAttempt?: Date; attempts: number;}
interface SyncError { timestamp: Date; error: string; code?: string; retryable: boolean;}Platform-Specific Mappings
Slack Integration
interface SlackPlatformData { // Original Slack message ts: string; // Slack timestamp (unique ID) thread_ts?: string; // Thread timestamp channel: string; // Channel ID team: string; // Workspace ID
// Slack-specific features blocks?: any[]; // Slack Block Kit attachments?: any[]; // Legacy attachments
// Metadata bot_id?: string; bot_profile?: any; client_msg_id?: string; subtype?: string; // message_changed, etc.
// Files files?: SlackFile[];
// Reactions (Slack format) slack_reactions?: { name: string; users: string[]; count: number; }[];}
interface SlackFile { id: string; name: string; title: string; mimetype: string; filetype: string; size: number; url_private: string; url_private_download: string; thumb_360?: string; thumb_480?: string; permalink: string;}Slack → Standard Mapping
function slackToStandard(slackMessage: any): StandardMessage { return { id: generateUUID(), platformId: slackMessage.ts, platform: MessagePlatform.SLACK,
content: { type: determineSlackMessageType(slackMessage), text: slackMessage.text, markdown: convertSlackMarkdown(slackMessage.text), blocks: convertSlackBlocks(slackMessage.blocks), attachments: convertSlackFiles(slackMessage.files) },
author: convertSlackUser(slackMessage.user),
conversation: { id: generateConversationId(slackMessage.channel), platformId: slackMessage.channel, platform: MessagePlatform.SLACK, type: determineSlackChannelType(slackMessage.channel), // ... other fields },
thread: slackMessage.thread_ts ? { id: generateThreadId(slackMessage.thread_ts), platformId: slackMessage.thread_ts, parentMessageId: slackMessage.thread_ts, rootMessageId: slackMessage.thread_ts, replyCount: slackMessage.reply_count || 0, participantCount: slackMessage.reply_users?.length || 0, lastReplyAt: slackMessage.latest_reply ? new Date(parseFloat(slackMessage.latest_reply) * 1000) : undefined } : undefined,
timestamp: new Date(parseFloat(slackMessage.ts) * 1000), editedTimestamp: slackMessage.edited ? new Date(parseFloat(slackMessage.edited.ts) * 1000) : undefined,
reactions: convertSlackReactions(slackMessage.reactions), mentions: extractSlackMentions(slackMessage.text),
metadata: { source: 'api', version: 1, syncStatus: { synced: true, direction: 'inbound', attempts: 1 }, isPinned: false, isStarred: false, isImportant: false, isEdited: !!slackMessage.edited, isDeleted: false, searchableText: normalizeForSearch(slackMessage.text), tags: [], custom: {} },
platformSpecific: { slack: { ts: slackMessage.ts, thread_ts: slackMessage.thread_ts, channel: slackMessage.channel, team: slackMessage.team, blocks: slackMessage.blocks, client_msg_id: slackMessage.client_msg_id, subtype: slackMessage.subtype, bot_id: slackMessage.bot_id } } };}Microsoft Teams Integration
interface TeamsPlatformData { // Original Teams message id: string; // Graph API message ID chatId?: string; // For 1:1 or group chats channelId?: string; // For channel messages teamId?: string; // Team ID
// Content body: { content: string; contentType: 'text' | 'html'; };
// Metadata messageType: string; // message, systemEventMessage, etc. importance: 'normal' | 'high' | 'urgent'; eventDetail?: any; // For system events
// Hosted content hostedContents?: TeamsHostedContent[];
// Mentions (Teams format) mentions?: TeamsMention[];
// Policy violation policyViolation?: any;}
interface TeamsHostedContent { id: string; contentBytes: string; contentType: string;}
interface TeamsMention { id: number; mentionText: string; mentioned: { user?: { id: string; displayName: string; userIdentityType: string; }; };}Teams → Standard Mapping
function teamsToStandard(teamsMessage: any): StandardMessage { return { id: generateUUID(), platformId: teamsMessage.id, platform: MessagePlatform.TEAMS,
content: { type: determineTeamsMessageType(teamsMessage), text: stripHtml(teamsMessage.body.content), html: teamsMessage.body.contentType === 'html' ? teamsMessage.body.content : undefined, markdown: convertHtmlToMarkdown(teamsMessage.body.content), blocks: parseTeamsContentBlocks(teamsMessage.body.content), attachments: convertTeamsAttachments(teamsMessage.attachments) },
author: convertTeamsUser(teamsMessage.from),
conversation: { id: generateConversationId(teamsMessage.chatId || teamsMessage.channelId), platformId: teamsMessage.chatId || teamsMessage.channelId, platform: MessagePlatform.TEAMS, type: teamsMessage.chatId ? ConversationType.GROUP : ConversationType.CHANNEL, // ... other fields },
thread: teamsMessage.replyToId ? { id: generateThreadId(teamsMessage.replyToId), platformId: teamsMessage.replyToId, parentMessageId: teamsMessage.replyToId, rootMessageId: teamsMessage.replyToId, replyCount: 0, // Would need separate API call participantCount: 0, lastReplyAt: undefined } : undefined,
timestamp: new Date(teamsMessage.createdDateTime), editedTimestamp: teamsMessage.lastModifiedDateTime ? new Date(teamsMessage.lastModifiedDateTime) : undefined, deletedTimestamp: teamsMessage.deletedDateTime ? new Date(teamsMessage.deletedDateTime) : undefined,
reactions: convertTeamsReactions(teamsMessage.reactions), mentions: convertTeamsMentions(teamsMessage.mentions),
metadata: { source: 'api', version: 1, syncStatus: { synced: true, direction: 'inbound', attempts: 1 }, isPinned: false, isStarred: false, isImportant: teamsMessage.importance !== 'normal', isEdited: teamsMessage.lastEditedDateTime !== undefined, isDeleted: teamsMessage.deletedDateTime !== undefined, searchableText: normalizeForSearch(stripHtml(teamsMessage.body.content)), tags: [], custom: {} },
platformSpecific: { teams: { id: teamsMessage.id, chatId: teamsMessage.chatId, channelId: teamsMessage.channelId, teamId: teamsMessage.teamId, importance: teamsMessage.importance, messageType: teamsMessage.messageType, webUrl: teamsMessage.webUrl, eventDetail: teamsMessage.eventDetail } } };}Database Schema
PostgreSQL Schema
-- Messages tableCREATE TABLE messages ( id UUID PRIMARY KEY DEFAULT gen_random_uuid(), platform_id VARCHAR(255) NOT NULL, platform VARCHAR(50) NOT NULL,
-- Content content_type VARCHAR(50) NOT NULL, content_text TEXT, content_html TEXT, content_markdown TEXT, content_blocks JSONB,
-- Author author_id UUID REFERENCES users(id),
-- Conversation conversation_id UUID REFERENCES conversations(id), thread_id UUID REFERENCES threads(id),
-- Timestamps timestamp TIMESTAMPTZ NOT NULL, edited_timestamp TIMESTAMPTZ, deleted_timestamp TIMESTAMPTZ,
-- Metadata is_pinned BOOLEAN DEFAULT FALSE, is_starred BOOLEAN DEFAULT FALSE, is_important BOOLEAN DEFAULT FALSE, is_edited BOOLEAN DEFAULT FALSE, is_deleted BOOLEAN DEFAULT FALSE,
searchable_text TEXT, tags TEXT[],
-- Platform-specific data platform_specific JSONB,
-- Sync sync_status VARCHAR(50) DEFAULT 'synced', last_synced_at TIMESTAMPTZ, sync_errors JSONB,
created_at TIMESTAMPTZ DEFAULT NOW(), updated_at TIMESTAMPTZ DEFAULT NOW(),
-- Indexes UNIQUE(platform, platform_id));
CREATE INDEX idx_messages_platform ON messages(platform);CREATE INDEX idx_messages_conversation ON messages(conversation_id);CREATE INDEX idx_messages_author ON messages(author_id);CREATE INDEX idx_messages_timestamp ON messages(timestamp DESC);CREATE INDEX idx_messages_thread ON messages(thread_id);CREATE INDEX idx_messages_search ON messages USING GIN(to_tsvector('english', searchable_text));CREATE INDEX idx_messages_platform_specific ON messages USING GIN(platform_specific);
-- Users tableCREATE TABLE users ( id UUID PRIMARY KEY DEFAULT gen_random_uuid(), platform_id VARCHAR(255) NOT NULL, platform VARCHAR(50) NOT NULL,
username VARCHAR(255) NOT NULL, display_name VARCHAR(255) NOT NULL, email VARCHAR(255), avatar VARCHAR(500),
is_bot BOOLEAN DEFAULT FALSE, is_guest BOOLEAN DEFAULT FALSE, roles TEXT[],
status_state VARCHAR(50), status_message TEXT, status_emoji VARCHAR(100), status_expires_at TIMESTAMPTZ,
timezone VARCHAR(100),
platform_specific JSONB,
created_at TIMESTAMPTZ DEFAULT NOW(), updated_at TIMESTAMPTZ DEFAULT NOW(),
UNIQUE(platform, platform_id));
CREATE INDEX idx_users_platform ON users(platform);CREATE INDEX idx_users_username ON users(username);CREATE INDEX idx_users_email ON users(email);
-- Conversations tableCREATE TABLE conversations ( id UUID PRIMARY KEY DEFAULT gen_random_uuid(), platform_id VARCHAR(255) NOT NULL, platform VARCHAR(50) NOT NULL,
type VARCHAR(50) NOT NULL, name VARCHAR(255), topic TEXT,
is_private BOOLEAN DEFAULT FALSE, is_archived BOOLEAN DEFAULT FALSE,
created_by UUID REFERENCES users(id), created_at TIMESTAMPTZ NOT NULL,
platform_specific JSONB,
updated_at TIMESTAMPTZ DEFAULT NOW(),
UNIQUE(platform, platform_id));
CREATE INDEX idx_conversations_platform ON conversations(platform);CREATE INDEX idx_conversations_type ON conversations(type);
-- Threads tableCREATE TABLE threads ( id UUID PRIMARY KEY DEFAULT gen_random_uuid(), platform_id VARCHAR(255) NOT NULL, platform VARCHAR(50) NOT NULL,
conversation_id UUID REFERENCES conversations(id), parent_message_id UUID REFERENCES messages(id), root_message_id UUID REFERENCES messages(id),
reply_count INTEGER DEFAULT 0, participant_count INTEGER DEFAULT 0, last_reply_at TIMESTAMPTZ,
created_at TIMESTAMPTZ DEFAULT NOW(), updated_at TIMESTAMPTZ DEFAULT NOW(),
UNIQUE(platform, platform_id));
CREATE INDEX idx_threads_conversation ON threads(conversation_id);CREATE INDEX idx_threads_root_message ON threads(root_message_id);
-- Attachments tableCREATE TABLE attachments ( id UUID PRIMARY KEY DEFAULT gen_random_uuid(), message_id UUID REFERENCES messages(id) ON DELETE CASCADE,
type VARCHAR(50) NOT NULL, filename VARCHAR(500), mime_type VARCHAR(100), size BIGINT,
url VARCHAR(1000), thumbnail_url VARCHAR(1000), preview_url VARCHAR(1000),
width INTEGER, height INTEGER, duration INTEGER,
title VARCHAR(500), description TEXT,
platform_specific JSONB,
created_at TIMESTAMPTZ DEFAULT NOW());
CREATE INDEX idx_attachments_message ON attachments(message_id);CREATE INDEX idx_attachments_type ON attachments(type);
-- Reactions tableCREATE TABLE reactions ( id UUID PRIMARY KEY DEFAULT gen_random_uuid(), message_id UUID REFERENCES messages(id) ON DELETE CASCADE,
emoji VARCHAR(100) NOT NULL, emoji_id VARCHAR(255), count INTEGER DEFAULT 1,
created_at TIMESTAMPTZ DEFAULT NOW(), updated_at TIMESTAMPTZ DEFAULT NOW(),
UNIQUE(message_id, emoji));
CREATE INDEX idx_reactions_message ON reactions(message_id);
-- Reaction users (many-to-many)CREATE TABLE reaction_users ( reaction_id UUID REFERENCES reactions(id) ON DELETE CASCADE, user_id UUID REFERENCES users(id) ON DELETE CASCADE, created_at TIMESTAMPTZ DEFAULT NOW(),
PRIMARY KEY (reaction_id, user_id));
-- Mentions tableCREATE TABLE mentions ( id UUID PRIMARY KEY DEFAULT gen_random_uuid(), message_id UUID REFERENCES messages(id) ON DELETE CASCADE,
type VARCHAR(50) NOT NULL, user_id UUID REFERENCES users(id), text VARCHAR(255) NOT NULL, start_position INTEGER NOT NULL, end_position INTEGER NOT NULL,
created_at TIMESTAMPTZ DEFAULT NOW());
CREATE INDEX idx_mentions_message ON mentions(message_id);CREATE INDEX idx_mentions_user ON mentions(user_id);CREATE INDEX idx_mentions_type ON mentions(type);API Design
REST API Endpoints
// Get messagesGET /api/messages ?conversationId={uuid} &platform={slack|teams|all} &after={timestamp} &before={timestamp} &limit={number} &includeDeleted={boolean}
// Get single messageGET /api/messages/:id
// Search messagesGET /api/messages/search ?q={query} &conversationId={uuid} &platform={slack|teams|all} &author={userId} &after={timestamp} &before={timestamp}
// Send messagePOST /api/messagesBody: { platform: 'slack', conversationId: 'uuid', content: { type: 'text', text: 'Hello' }, threadId?: 'uuid'}
// Edit messagePATCH /api/messages/:idBody: { content: { text: 'Updated text' }}
// Delete messageDELETE /api/messages/:id
// React to messagePOST /api/messages/:id/reactionsBody: { emoji: '👍'}
// Get conversationsGET /api/conversations ?platform={slack|teams|all} &type={direct|group|channel}
// Get conversation detailsGET /api/conversations/:id
// Get conversation messagesGET /api/conversations/:id/messages ?after={timestamp} &limit={number}GraphQL Schema
type Message { id: ID! platformId: String! platform: MessagePlatform!
content: MessageContent! author: User! conversation: Conversation! thread: Thread
timestamp: DateTime! editedTimestamp: DateTime deletedTimestamp: DateTime
reactions: [Reaction!]! mentions: [Mention!]!
metadata: MessageMetadata!}
type MessageContent { type: MessageType! text: String html: String markdown: String blocks: [ContentBlock!] attachments: [Attachment!]!}
type User { id: ID! platformId: String! platform: MessagePlatform! username: String! displayName: String! email: String avatar: String isBot: Boolean! status: UserStatus}
type Conversation { id: ID! platformId: String! platform: MessagePlatform! type: ConversationType! name: String topic: String participants: [User!]! isPrivate: Boolean! isArchived: Boolean! messages( after: DateTime before: DateTime limit: Int ): [Message!]!}
type Query { message(id: ID!): Message messages( conversationId: ID platform: MessagePlatform after: DateTime before: DateTime limit: Int ): [Message!]!
searchMessages( query: String! conversationId: ID platform: MessagePlatform authorId: ID ): [Message!]!
conversation(id: ID!): Conversation conversations( platform: MessagePlatform type: ConversationType ): [Conversation!]!
user(id: ID!): User}
type Mutation { sendMessage(input: SendMessageInput!): Message! editMessage(id: ID!, content: MessageContentInput!): Message! deleteMessage(id: ID!): Boolean! addReaction(messageId: ID!, emoji: String!): Reaction! removeReaction(messageId: ID!, emoji: String!): Boolean!}
input SendMessageInput { platform: MessagePlatform! conversationId: ID! content: MessageContentInput! threadId: ID}
input MessageContentInput { type: MessageType! text: String attachments: [AttachmentInput!]}Implementation Notes
1. Message Normalization
- Text normalization: Convert platform-specific markdown/formatting to standard format
- Timestamp handling: All platforms use different timestamp formats (Unix, ISO8601, etc.)
- ID generation: Use UUIDs for internal IDs, preserve platform IDs for sync
2. Real-time Sync
- Use webhooks/event subscriptions for real-time updates
- Implement event queue (e.g., Redis/BullMQ) for processing
- Handle rate limits and backoff strategies
- Implement idempotency for duplicate events
3. Search & Indexing
- Full-text search using PostgreSQL
tsvectoror Elasticsearch - Index both normalized and platform-specific content
- Support fuzzy matching and relevance scoring
4. Conflict Resolution
- Last-write-wins for simple conflicts
- Preserve edit history for audit trail
- Handle platform-specific constraints (e.g., message length limits)
5. Privacy & Security
- Respect platform permissions and visibility
- Implement field-level encryption for sensitive data
- Audit logging for all message access
- GDPR compliance for message retention
Migration Strategy
Phase 1: Read-Only Integration
- Implement platform adapters for Slack and Teams
- Build message ingestion pipeline
- Normalize and store messages
- Build query API
Phase 2: Bi-directional Sync
- Implement write operations (send, edit, delete)
- Add real-time event processing
- Implement conflict resolution
- Add retry and error handling
Phase 3: Advanced Features
- Search and analytics
- Message threading and relationships
- Rich media handling
- Cross-platform notifications
Testing Strategy
Unit Tests
- Platform adapter transformations
- Message normalization logic
- Search query building
- ID generation and mapping
Integration Tests
- End-to-end message flow from platform to database
- API endpoints with real data
- Real-time event processing
- Cross-platform message sending
Platform-Specific Tests
- Slack Block Kit rendering
- Teams HTML content parsing
- Attachment/file handling
- Reaction and mention extraction