Chapter 3

Neo4j Deep Dive

Neo4j is the world's leading graph database platform, pioneering the property graph model and the Cypher query language. With over a decade of development and millions of deployments worldwide, Neo4j has become synonymous with graph databases. This chapter explores Neo4j's architecture, features, and best practices for building production-grade applications.

Introduction to Neo4j

Founded in 2007, Neo4j was built from the ground up as a native graph database. Unlike databases that added graph capabilities later, Neo4j's entire architecture is optimized for graph storage and traversal. This native approach delivers unmatched performance for connected data queries.

Key Features

ACID Transactions: Full transactional support with rollback capabilities
Cypher Query Language: Intuitive, SQL-like declarative language for graphs
Index-Free Adjacency: Native graph storage for O(1) traversals
High Availability: Clustering and replication for mission-critical applications
Graph Algorithms: Built-in library of 50+ production-quality algorithms
APOC Procedures: Extensive library of utility functions and procedures
Visualization Tools: Browser-based interface for exploration and development
Multi-Language Support: Drivers for Java, Python, JavaScript, .NET, Go, and more

Neo4j Architecture

Understanding Neo4j's architecture helps you build better applications:

Storage Layer

Neo4j uses a custom storage format optimized for graphs:

Fixed-Size Records: Nodes and relationships stored in fixed-size records for fast random access
Property Stores: Separate stores for different property types (strings, arrays, etc.)
Index-Free Adjacency: Each node physically stores references to its relationships
Memory Mapping: Leverages OS page cache for hot data

Transaction Layer

Neo4j provides full ACID guarantees:

// Begin transaction
BEGIN
MATCH (p:Person {name: 'Alice'})
SET p.balance = p.balance - 100

MATCH (q:Person {name: 'Bob'})
SET q.balance = q.balance + 100
COMMIT

// Or rollback on error
ROLLBACK

Query Execution Engine

Cypher queries are compiled into execution plans:

// View execution plan
EXPLAIN
MATCH (p:Person)-[:KNOWS*2]->(friend)
WHERE p.name = 'Alice'
RETURN friend.name

// Profile with actual statistics
PROFILE
MATCH (p:Person)-[:KNOWS*2]->(friend)
WHERE p.name = 'Alice'
RETURN friend.name

Neo4j Editions

Edition	Use Case	Key Features	License
Community	Development, Small Projects	Single instance, Core features	GPLv3
Enterprise	Production, Enterprise	Clustering, Hot backups, Advanced monitoring	Commercial
AuraDB	Cloud, Managed Service	Fully managed, Auto-scaling, Built-in security	Commercial
Desktop	Local Development	GUI management, Plugin ecosystem	Free

Installation and Setup

Neo4j Desktop (Recommended for Learning)

Download from neo4j.com/download
Install and launch Neo4j Desktop
Create a new project and database
Start the database and open Neo4j Browser

Docker Installation

# Pull Neo4j image
docker pull neo4j:latest

# Run Neo4j container
docker run \
    --name neo4j \
    -p 7474:7474 -p 7687:7687 \
    -e NEO4J_AUTH=neo4j/password \
    -v $HOME/neo4j/data:/data \
    neo4j:latest

# Access at http://localhost:7474

Server Installation (Ubuntu/Debian)

# Add Neo4j repository
wget -O - https://debian.neo4j.com/neotechnology.gpg.key | sudo apt-key add -
echo 'deb https://debian.neo4j.com stable latest' | sudo tee /etc/apt/sources.list.d/neo4j.list

# Install Neo4j
sudo apt-get update
sudo apt-get install neo4j

# Start service
sudo systemctl start neo4j
sudo systemctl enable neo4j

Neo4j Browser

Neo4j Browser is a web-based interface for querying and visualizing your graph:

Key Features

Query Editor: Write and execute Cypher queries with syntax highlighting
Graph Visualization: Interactive visualization of query results
Database Inspection: Browse node labels, relationship types, and properties
Built-in Guides: Interactive tutorials and documentation
Query History: Review and re-run previous queries

Useful Browser Commands

// Show database schema
:schema

// Show database statistics
:sysinfo

// List available procedures
CALL dbms.procedures()

// Show all indexes
:schema indexes

// Clear the graph visualization
:clear

Configuration and Tuning

Neo4j's configuration file (neo4j.conf) controls database behavior:

Memory Settings

# Page cache (OS-level file system cache)
dbms.memory.pagecache.size=4G

# Heap memory for query execution
dbms.memory.heap.initial_size=2G
dbms.memory.heap.max_size=4G

# Transaction log size
dbms.tx_log.rotation.retention_policy=2G size

Network Configuration

# HTTP connector
dbms.connector.http.enabled=true
dbms.connector.http.listen_address=0.0.0.0:7474

# Bolt connector (native protocol)
dbms.connector.bolt.enabled=true
dbms.connector.bolt.listen_address=0.0.0.0:7687

Security Settings

# Authentication and authorization
dbms.security.auth_enabled=true

# Encryption
dbms.ssl.policy.bolt.enabled=true
dbms.ssl.policy.bolt.base_directory=certificates/bolt

# Password policy
dbms.security.auth_minimum_password_length=8

Indexes and Constraints

Proper indexing is crucial for query performance:

Creating Indexes

// Single-property index
CREATE INDEX person_email FOR (p:Person) ON (p.email)

// Composite index
CREATE INDEX person_name_city FOR (p:Person) ON (p.lastName, p.city)

// Full-text index
CREATE FULLTEXT INDEX person_search FOR (p:Person) ON EACH [p.name, p.bio]

// Using full-text search
CALL db.index.fulltext.queryNodes("person_search", "Alice engineer") 
YIELD node, score
RETURN node.name, score

Constraints

// Unique constraint (automatically creates index)
CREATE CONSTRAINT person_email_unique FOR (p:Person) REQUIRE p.email IS UNIQUE

// Node property existence
CREATE CONSTRAINT person_name_exists FOR (p:Person) REQUIRE p.name IS NOT NULL

// Relationship property existence
CREATE CONSTRAINT knows_since_exists FOR ()-[k:KNOWS]-() REQUIRE k.since IS NOT NULL

// Node key (multiple properties must be unique together)
CREATE CONSTRAINT person_key FOR (p:Person) REQUIRE (p.firstName, p.lastName, p.birthDate) IS NODE KEY

Data Import Strategies

LOAD CSV for Small to Medium Datasets

// Import from CSV file
LOAD CSV WITH HEADERS FROM 'file:///people.csv' AS row
CREATE (p:Person {
    id: toInteger(row.id),
    name: row.name,
    email: row.email,
    age: toInteger(row.age)
})

// Using periodic commit for large files
:auto USING PERIODIC COMMIT 500
LOAD CSV WITH HEADERS FROM 'file:///large_dataset.csv' AS row
CREATE (p:Person {id: toInteger(row.id), name: row.name})

neo4j-admin import for Large Datasets

# Bulk import from CSV (database must be stopped)
neo4j-admin database import full \
  --nodes=Person=people.csv \
  --nodes=Company=companies.csv \
  --relationships=WORKS_FOR=employment.csv \
  --delimiter="," \
  neo4j

APOC for Complex Imports

// Import from JSON
CALL apoc.load.json('https://api.example.com/users') YIELD value
CREATE (p:Person {
    id: value.id,
    name: value.name,
    email: value.email
})

// Import from database
CALL apoc.load.jdbc('jdbc:mysql://localhost/mydb', 'SELECT * FROM users') YIELD row
CREATE (p:Person {id: row.id, name: row.name})

Backup and Recovery

Online Backup (Enterprise)

# Perform online backup
neo4j-admin database backup --to-path=/backups neo4j

# Restore from backup
neo4j-admin database restore --from-path=/backups/neo4j neo4j

Dump and Load (All Editions)

# Dump database
neo4j-admin database dump --to-path=/dumps neo4j

# Load dump
neo4j-admin database load --from-path=/dumps/neo4j.dump neo4j --overwrite-destination=true

Monitoring and Observability

Query Logging

# Enable query logging in neo4j.conf
dbms.logs.query.enabled=true
dbms.logs.query.threshold=1s
dbms.logs.query.parameter_logging_enabled=true

Metrics and Monitoring

// Database statistics
CALL dbms.queryJmx("org.neo4j:*") YIELD name, attributes

// List running queries
CALL dbms.listQueries()

// Kill a long-running query
CALL dbms.killQuery("query-123")

// Transaction statistics
CALL dbms.listTransactions()

Integration with Monitoring Tools

Neo4j exposes metrics via JMX and can integrate with:

Prometheus: For metrics collection and alerting
Grafana: For visualization and dashboards
Elasticsearch: For log aggregation and analysis

Neo4j Drivers and Client Applications

Official Drivers

Neo4j provides official drivers for major languages:

Python Driver

from neo4j import GraphDatabase

class Neo4jConnection:
    def __init__(self, uri, user, password):
        self.driver = GraphDatabase.driver(uri, auth=(user, password))
    
    def close(self):
        self.driver.close()
    
    def query(self, query, parameters=None):
        with self.driver.session() as session:
            result = session.run(query, parameters)
            return [record for record in result]

# Usage
conn = Neo4jConnection("bolt://localhost:7687", "neo4j", "password")
results = conn.query("""
    MATCH (p:Person)-[:KNOWS]->(friend)
    WHERE p.name = $name
    RETURN friend.name as friendName
""", {"name": "Alice"})

for record in results:
    print(record["friendName"])

conn.close()

JavaScript Driver

const neo4j = require('neo4j-driver');

const driver = neo4j.driver(
    'bolt://localhost:7687',
    neo4j.auth.basic('neo4j', 'password')
);

async function findFriends(name) {
    const session = driver.session();
    try {
        const result = await session.run(
            'MATCH (p:Person {name: $name})-[:KNOWS]->(friend) RETURN friend.name as friendName',
            { name }
        );
        return result.records.map(r => r.get('friendName'));
    } finally {
        await session.close();
    }
}

// Usage
findFriends('Alice').then(friends => {
    console.log(friends);
    driver.close();
});

High Availability and Clustering

Neo4j Enterprise supports clustering for production environments:

Causal Clustering

Neo4j's clustering architecture uses a Raft-based consensus protocol:

Core Servers: Participate in consensus, handle writes
Read Replicas: Serve read-only queries, scale horizontally
Automatic Failover: Leader election on failure

# Core server configuration
dbms.mode=CORE
causal_clustering.initial_discovery_members=server1:5000,server2:5000,server3:5000
causal_clustering.minimum_core_cluster_size_at_formation=3

# Read replica configuration
dbms.mode=READ_REPLICA
causal_clustering.initial_discovery_members=server1:5000,server2:5000,server3:5000

Performance Best Practices

Use Indexes: Create indexes on frequently queried properties
Parameterize Queries: Use parameters to enable query plan caching
Limit Result Sets: Use LIMIT to avoid returning excessive data
Avoid Cartesian Products: Ensure patterns are properly connected
Use PROFILE: Analyze query execution plans to identify bottlenecks
Batch Operations: Group multiple operations in single transactions
Monitor Memory: Ensure adequate page cache and heap size
Use Appropriate Data Types: Store dates as temporal types, not strings

Chapter Summary

Neo4j is a mature, production-ready graph database platform with extensive tooling, drivers, and ecosystem support. From simple local development with Neo4j Desktop to highly available clustered deployments in the enterprise, Neo4j provides the foundation for building graph-powered applications. Understanding its architecture, configuration options, and best practices enables you to build performant, scalable solutions.

In the next chapter, we'll explore graph query languages in depth, mastering Cypher and comparing it with alternatives like Gremlin.

Chapter 3 — Notes & References

WIA Standards Public Repository (graph-database folder), MIT License, GitHub: WIA-Official/wia-standards-public/tree/main/graph-database — open standard initiative providing source code for simulator, spec, API, and ebook assets cited throughout this volume; serves as the canonical verification record for all primary-source citations made by the WIA standard committee in this chapter. Canonical ENUM tokens used in this volume include NEO4J, NEPTUNE, DGRAPH, ARANGODB, JANUSGRAPH, TIGERGRAPH, MEMGRAPH, NEBULA_GRAPH, CYPHER, GREMLIN, SPARQL, GQL, PAGERANK, LOUVAIN, NODE2VEC, GRAPHSAGE, ISO_GQL_2024, W3C_RDF, W3C_SPARQL, LDBC_SNB, WIKIDATA, DBPEDIA, YAGO, CONCEPTNET.