๐Ÿ—„๏ธ BioDB
HOME

๐Ÿง  Database Fundamentals

๐ŸŽฏ What is a Biological Database?

A biological database is an organized collection of biological data (sequences, structures, pathways, expression) that is stored, managed, and made accessible for querying and analysis. Biological databases are essential for storing the massive amounts of data generated by genomics, proteomics, and structural biology. They enable researchers to retrieve, compare, and analyze biological information.

Why Databases Matter

Storage: Massive data. Retrieval: Fast access. Integration: Cross-reference. Annotation: Curated knowledge. Analysis: Comparative. Reproducibility: Standardized.

๐Ÿ“Œ Storage๐Ÿ“Œ Retrieval๐Ÿ“Œ Integration๐Ÿ“Œ Annotation
# Biological Databases โ€” the big picture # Organized collections of biological data. print(""" === Biological Databases === What they are: - Organized collections of biological data - Sequences, structures, pathways, expression - Stored, managed, accessible - Enable retrieval and analysis Why they matter: - Storage of massive data - Fast retrieval - Cross-referencing - Curated annotation - Comparative analysis - Reproducibility Major databases: - NCBI (GenBank, RefSeq, PubMed) - UniProt (protein sequences) - Ensembl (genomes) - PDB (protein structures) - KEGG (pathways) - Reactome (pathways) - GEO (expression) - Pfam (protein families) - InterPro (protein domains) Applications: - Sequence retrieval - BLAST search - Phylogenetics - Structure analysis - Pathway analysis - Expression analysis """) print("=== Historical Context ===") print(""" 1960s: First protein sequence databases (Atlas of Protein Sequence) 1970s: PDB (1971), GenBank (1982) 1980s: EMBL, Swiss-Prot 1990s: NCBI, Entrez, KEGG 2000s: Ensembl, UniProt, GEO 2010s: Big data, cloud, APIs 2020s: AI, knowledge graphs, integration """) print("=== Why It's Hard ===") print(""" 1. DATA VOLUME - Petabytes of data - Growing exponentially 2. DATA HETEROGENEITY - Different formats - Different standards 3. DATA QUALITY - Errors - Redundancy - Incomplete annotation 4. INTEGRATION - Cross-referencing - Identifiers - Ontologies 5. ACCESS - APIs - Rate limits - Authentication """)

๐ŸŽฏ Why Databases Matter

๐ŸŽฏ The Value of Biological Databases

Biological databases are the backbone of modern biology. They store, organize, and make accessible the data that drives discovery.

Key Benefits

Storage: Petabytes. Retrieval: Fast. Integration: Cross-ref. Annotation: Curated. Analysis: Comparative. Sharing: Standardized.

๐ŸŽฏ Storage๐ŸŽฏ Retrieval๐ŸŽฏ Integration๐ŸŽฏ Annotation
# Why Databases Matter print("=== 1. Data Storage ===") print(""" Storage: - GenBank: > 3 billion sequences - UniProt: > 200 million proteins - PDB: > 200,000 structures - GEO: > 5 million samples Growth: - Exponential - Doubling every ~2 years - Big data challenge """) print("\n=== 2. Fast Retrieval ===") print(""" Retrieval: - Query by ID - Query by sequence - Query by keyword - BLAST search Example: Entrez: search NCBI UniProt: search proteins PDB: search structures """) print("\n=== 3. Cross-Referencing ===") print(""" Cross-referencing: - Link databases - Same entity, different DBs - Integration Example: UniProt โ†’ PDB (structure) UniProt โ†’ Ensembl (genome) UniProt โ†’ KEGG (pathway) GenBank โ†’ PubMed (literature) """) print("\n=== 4. Curated Annotation ===") print(""" Annotation: - Manual curation - Automated annotation - Quality control Example: Swiss-Prot (curated) RefSeq (curated) Pfam (curated) """) print("\n=== 5. Comparative Analysis ===") print(""" Comparative: - BLAST - Multiple alignment - Phylogenetics - Structure comparison Example: BLAST against GenBank โ†’ Find homologs """) print("\n=== 6. Reproducibility ===") print(""" Reproducibility: - Standardized data - Versioning - Accession numbers - Citation Example: GenBank accession: NC_000001 โ†’ Same sequence everywhere """) print("\n=== 7. Knowledge Discovery ===") print(""" Discovery: - Data mining - Machine learning - Knowledge graphs - Drug discovery Example: Drug target identification โ†’ Database mining """) print("\n=== 8. Clinical Applications ===") print(""" Clinical: - Variant interpretation - Disease genes - Pharmacogenomics Example: ClinVar โ†’ Clinical variants โ†’ Diagnosis """) print("\n=== 9. Education ===") print(""" Education: - Teaching - Training - Resources Example: NCBI tutorials UniProt tutorials PDB tutorials """) print("\n=== 10. Research ===") print(""" Research: - Hypothesis generation - Data analysis - Publication Example: Database search โ†’ New hypothesis โ†’ Experiment โ†’ Publication """)

๐Ÿ“Š Types of Databases

๐ŸŽฏ Database Classification

Biological databases can be classified in several ways: by data type, by curation level, by scope, and by access method.

Types of Databases

Primary: Raw data. Secondary: Curated. Composite: Integrated. Specialized: Specific. Sequence: DNA/protein. Structure: 3D.

๐Ÿ“Š Primary๐Ÿ“Š Secondary๐Ÿ“Š Composite๐Ÿ“Š Specialized
# Types of Databases print("=== 1. Primary Databases ===") print(""" Primary: - Raw experimental data - Submitted by researchers - May be redundant - May have errors Examples: - GenBank (DNA) - PDB (structures) - GEO (expression) - SRA (reads) - PRIDE (proteomics) Characteristics: - Direct submission - No curation - Redundant """) print("\n=== 2. Secondary Databases ===") print(""" Secondary: - Derived from primary - Curated - Annotated - Non-redundant Examples: - UniProt (proteins) - RefSeq (reference) - Pfam (families) - InterPro (domains) - KEGG (pathways) Characteristics: - Curated - Annotated - Integrated """) print("\n=== 3. Composite Databases ===") print(""" Composite: - Integrate multiple sources - Unified interface - Cross-referenced Examples: - Entrez (NCBI) - Ensembl (genomes) - UCSC (genomes) - InterPro (domains) Characteristics: - Integration - Cross-referencing - Unified query """) print("\n=== 4. Specialized Databases ===") print(""" Specialized: - Specific data type - Specific organism - Specific disease Examples: - FlyBase (Drosophila) - TAIR (Arabidopsis) - SGD (yeast) - OMIM (human disease) - ClinVar (clinical variants) Characteristics: - Focused - Deep annotation - Community-specific """) print("\n=== 5. Sequence Databases ===") print(""" Sequence: - DNA, RNA, protein - Primary + secondary Examples: - GenBank (DNA) - RefSeq (reference) - UniProt (protein) - EMBL (DNA) - DDBJ (DNA) Types: - Nucleotide - Protein - Genome """) print("\n=== 6. Structure Databases ===") print(""" Structure: - 3D structures - Experimental + predicted Examples: - PDB (experimental) - AlphaFold DB (predicted) - CATH (classification) - SCOP (classification) - EMDB (EM) Types: - X-ray - NMR - Cryo-EM - Predicted """) print("\n=== 7. Pathway Databases ===") print(""" Pathway: - Metabolic pathways - Signaling pathways - Regulatory networks Examples: - KEGG - Reactome - WikiPathways - BioCyc - MetaCyc Types: - Metabolic - Signaling - Regulatory """) print("\n=== 8. Expression Databases ===") print(""" Expression: - Gene expression - Transcriptomics - Proteomics Examples: - GEO (microarray/RNA-seq) - ArrayExpress - GTEx (human tissues) - TCGA (cancer) - Human Protein Atlas Types: - Microarray - RNA-seq - Proteomics """) print("\n=== 9. Literature Databases ===") print(""" Literature: - Scientific papers - Abstracts - Citations Examples: - PubMed - PMC - arXiv - bioRxiv - Google Scholar Types: - Abstracts - Full text - Preprints """) print("\n=== 10. Classification ===") print(f"{'Type':<20} {'Description':<30} {'Examples':<30}") print("-" * 80) print(f"{'Primary':<20} {'Raw data':<30} {'GenBank, PDB':<30}") print(f"{'Secondary':<20} {'Curated':<30} {'UniProt, RefSeq':<30}") print(f"{'Composite':<20} {'Integrated':<30} {'Entrez, Ensembl':<30}") print(f"{'Specialized':<20} {'Organism/disease':<30} {'FlyBase, OMIM':<30}") print(f"{'Sequence':<20} {'DNA/protein':<30} {'GenBank, UniProt':<30}") print(f"{'Structure':<20} {'3D structures':<30} {'PDB, AlphaFold':<30}") print(f"{'Pathway':<20} {'Pathways':<30} {'KEGG, Reactome':<30}") print(f"{'Expression':<20} {'Expression':<30} {'GEO, GTEx':<30}") print(f"{'Literature':<20} {'Papers':<30} {'PubMed, PMC':<30}")

๐Ÿงฌ Nucleotide Databases

๐ŸŽฏ DNA and RNA Databases

Nucleotide databases store DNA and RNA sequences. They are essential for genomics, transcriptomics, and molecular biology.

Nucleotide Databases

GenBank: Primary. RefSeq: Curated. EMBL: Europe. DDBJ: Japan. Ensembl: Genomes. UCSC: Genomes.

๐Ÿงฌ GenBank๐Ÿงฌ RefSeq๐Ÿงฌ EMBL๐Ÿงฌ Ensembl
# Nucleotide Databases print("=== 1. GenBank ===") print(""" GenBank: - NCBI - Primary database - All submitted sequences - Redundant - > 3 billion sequences Access: - Web: ncbi.nlm.nih.gov/genbank - Entrez - FTP Formats: - GenBank - FASTA - XML Example: Accession: NC_000001 โ†’ Human chromosome 1 """) print("\n=== 2. RefSeq ===") print(""" RefSeq: - NCBI - Secondary database - Curated - Non-redundant - Reference sequences Access: - Web: ncbi.nlm.nih.gov/refseq - Entrez Prefixes: - NC_ (genome) - NM_ (mRNA) - NP_ (protein) - NR_ (non-coding) Example: NM_000546 โ†’ TP53 mRNA """) print("\n=== 3. EMBL ===") print(""" EMBL: - EBI (Europe) - Primary database - Part of INSDC - Nucleotide sequences Access: - Web: ebi.ac.uk/ena - API Partners: - GenBank (NCBI) - DDBJ (Japan) - INSDC (international) """) print("\n=== 4. DDBJ ===") print(""" DDBJ: - Japan - Primary database - Part of INSDC - Nucleotide sequences Access: - Web: ddbj.nig.ac.jp - API """) print("\n=== 5. Ensembl ===") print(""" Ensembl: - EBI + Wellcome Sanger - Genome database - Annotation - Comparative genomics Access: - Web: ensembl.org - REST API - FTP Features: - Genomes - Genes - Variants - Comparative - Regulation Example: ENSG00000141510 โ†’ TP53 gene """) print("\n=== 6. UCSC Genome Browser ===") print(""" UCSC: - University of California - Genome browser - Annotation - Comparative Access: - Web: genome.ucsc.edu - API Features: - Genomes - Tracks - Annotation - Comparative """) print("\n=== 7. Other Nucleotide Databases ===") print(""" 1. SILVA - rRNA - Microbial 2. Rfam - RNA families - Non-coding 3. miRBase - microRNA 4. GtRNAdb - tRNA 5. PlasmoDB - Plasmodium 6. VectorBase - Vectors """) print("\n=== 8. Accession Numbers ===") print(""" Accession: - Unique identifier - Stable - Versioned Examples: - NC_000001.11 (chromosome) - NM_000546.6 (mRNA) - NP_000537.3 (protein) - ENSG00000141510 (gene) Format: - Prefix - Number - Version """) print("\n=== 9. Format ===") print(""" GenBank format: LOCUS NC_000001 248956422 bp DNA DEFINITION Homo sapiens chromosome 1 ACCESSION NC_000001 VERSION NC_000001.11 KEYWORDS RefSeq SOURCE Homo sapiens FEATURES ... ORIGIN ... // FASTA format: >NC_000001.11 Homo sapiens chromosome 1 NNNNNNNNNNNNNNNNNNNN... """) print("\n=== 10. Use Cases ===") print(""" Use cases: - Sequence retrieval - BLAST search - Primer design - Phylogenetics - Variant analysis - Gene annotation - Comparative genomics """)

๐Ÿงช Protein Databases

๐ŸŽฏ Protein Sequence and Function

Protein databases store protein sequences, structures, families, and functional annotations. They are essential for proteomics, structural biology, and functional genomics.

Protein Databases

UniProt: Sequences. Pfam: Families. InterPro: Domains. PDB: Structures. STRING: Interactions. PROSITE: Motifs.

๐Ÿงช UniProt๐Ÿงช Pfam๐Ÿงช InterPro๐Ÿงช PDB
# Protein Databases print("=== 1. UniProt ===") print(""" UniProt: - Universal Protein Resource - Most comprehensive - Protein sequences - Functional annotation Components: - Swiss-Prot (curated) - TrEMBL (unreviewed) - UniParc (archive) Access: - Web: uniprot.org - REST API - FTP Example: P04637 โ†’ Human TP53 """) print("\n=== 2. Pfam ===") print(""" Pfam: - Protein families - HMM profiles - Domains - Curated Access: - Web: pfam.xfam.org - API - FTP Example: PF00001 โ†’ 7 transmembrane receptor """) print("\n=== 3. InterPro ===") print(""" InterPro: - Integrative - Protein families - Domains - Functional sites Access: - Web: ebi.ac.uk/interpro - API Members: - Pfam - PROSITE - PRINTS - SMART - PANTHER Example: IPR001841 โ†’ Zinc finger """) print("\n=== 4. PDB ===") print(""" PDB: - Protein Data Bank - 3D structures - Experimental - X-ray, NMR, Cryo-EM Access: - Web: rcsb.org - API - FTP Example: 1TUP โ†’ p53 DNA-binding domain """) print("\n=== 5. STRING ===") print(""" STRING: - Protein-protein interactions - Functional associations - Confidence scores Access: - Web: string-db.org - API Example: TP53 โ†’ Interaction network """) print("\n=== 6. PROSITE ===") print(""" PROSITE: - Protein motifs - Patterns - Profiles - Functional sites Access: - Web: prosite.expasy.org - API Example: PS00001 โ†’ N-glycosylation site """) print("\n=== 7. Other Protein Databases ===") print(""" 1. PRIDE - Proteomics - Mass spectrometry 2. PeptideAtlas - Peptides - Proteomics 3. Human Protein Atlas - Expression - Localization 4. neXtProt - Human proteins - Annotation 5. MEROPS - Proteases - Peptidases """) print("\n=== 8. UniProt Format ===") print(""" UniProt format: ID TP53_HUMAN Reviewed; 393 AA. AC P04637; A0A024R1X2; ... DT 01-AUG-1988, integrated into UniProtKB/Swiss-Prot. DE Cellular tumor antigen p53 GN Name=TP53; Synonyms=P53; OS Homo sapiens (Human). OX NCBI_TaxID=9606; ... SQ SEQUENCE 393 AA; MEEPQSDPSV EPPLSQETFS DLWKLLPENN VLSPLPSQAM DDLMLSPDDI ... // """) print("\n=== 9. Use Cases ===") print(""" Use cases: - Protein sequence retrieval - BLAST search - Domain analysis - Structure prediction - Interaction networks - Functional annotation - Phylogenetics - Drug discovery """) print("\n=== 10. Cross-References ===") print(""" Cross-references: - UniProt โ†’ PDB - UniProt โ†’ Ensembl - UniProt โ†’ KEGG - UniProt โ†’ Pfam - UniProt โ†’ InterPro - UniProt โ†’ STRING Example: UniProt P04637 โ†’ PDB: 1TUP, 2OCJ, ... โ†’ Ensembl: ENSG00000141510 โ†’ KEGG: hsa:7157 """)

๐ŸŒ Genome Databases

๐ŸŽฏ Whole Genome Data

Genome databases store complete genome sequences, annotations, and comparative data. They are essential for genomics, evolutionary biology, and precision medicine.

Genome Databases

Ensembl: Vertebrates. UCSC: Browser. NCBI: Genomes. JGI: Plants/microbes. TAIR: Arabidopsis. FlyBase: Drosophila.

๐ŸŒ Ensembl๐ŸŒ UCSC๐ŸŒ NCBI๐ŸŒ JGI
# Genome Databases print("=== 1. Ensembl ===") print(""" Ensembl: - EBI + Sanger - Vertebrate genomes - Annotation - Comparative genomics Access: - Web: ensembl.org - REST API - FTP Features: - Genomes - Genes - Transcripts - Variants - Regulation - Comparative Example: ENSG00000141510 โ†’ TP53 """) print("\n=== 2. UCSC Genome Browser ===") print(""" UCSC: - University of California - Genome browser - Many genomes - Tracks Access: - Web: genome.ucsc.edu - API - FTP Features: - Genomes - Tracks - Annotation - Comparative - Table Browser Example: hg38 โ†’ Human genome """) print("\n=== 3. NCBI Genome ===") print(""" NCBI Genome: - NCBI - All genomes - Assembly - Annotation Access: - Web: ncbi.nlm.nih.gov/genome - Entrez - FTP Features: - Assemblies - Annotation - BLAST - Downloads Example: GCF_000001405.40 โ†’ Human GRCh38 """) print("\n=== 4. JGI ===") print(""" JGI (Joint Genome Institute): - DOE - Plants - Microbes - Fungi Access: - Web: jgi.doe.gov - Phytozome - MycoCosm Features: - Genomes - Annotation - Comparative - Tools Example: Phytozome โ†’ Plant genomes """) print("\n=== 5. TAIR ===") print(""" TAIR: - Arabidopsis - Model plant - Annotation - Tools Access: - Web: arabidopsis.org Features: - Genome - Genes - Markers - Expression - Literature """) print("\n=== 6. FlyBase ===") print(""" FlyBase: - Drosophila - Model organism - Annotation - Tools Access: - Web: flybase.org Features: - Genome - Genes - Mutants - Expression - Literature """) print("\n=== 7. SGD ===") print(""" SGD (Saccharomyces Genome Database): - Yeast - Model organism - Annotation - Tools Access: - Web: yeastgenome.org Features: - Genome - Genes - Proteins - Interactions - Expression """) print("\n=== 8. WormBase ===") print(""" WormBase: - C. elegans - Model organism - Annotation - Tools Access: - Web: wormbase.org Features: - Genome - Genes - Expression - Phenotypes """) print("\n=== 9. Other Genome Databases ===") print(""" 1. ZFIN - Zebrafish 2. MGI - Mouse 3. RGD - Rat 4. Gramene - Grasses 5. Phytozome - Plants 6. Ensembl Genomes - Non-vertebrates 7. NCBI Assembly - All assemblies """) print("\n=== 10. Use Cases ===") print(""" Use cases: - Genome browsing - Gene retrieval - Variant analysis - Comparative genomics - Phylogenetics - Annotation - Primer design - CRISPR design """)

๐Ÿ—๏ธ Structure Databases

๐ŸŽฏ 3D Structural Data

Structure databases store 3D structures of biological macromolecules. They are essential for structural biology, drug discovery, and understanding molecular function.

Structure Databases

PDB: Experimental. AlphaFold: Predicted. EMDB: EM maps. CATH: Classification. SCOP: Classification. BMRB: NMR.

๐Ÿ—๏ธ PDB๐Ÿ—๏ธ AlphaFold๐Ÿ—๏ธ EMDB๐Ÿ—๏ธ CATH
# Structure Databases print("=== 1. PDB ===") print(""" PDB (Protein Data Bank): - Experimental structures - X-ray, NMR, Cryo-EM - > 200,000 structures - Worldwide Access: - Web: rcsb.org - API - FTP Formats: - PDB - mmCIF - PDBML Example: 1TUP โ†’ p53 DNA-binding domain """) print("\n=== 2. AlphaFold DB ===") print(""" AlphaFold DB: - Predicted structures - DeepMind + EBI - > 200 million - Proteomes Access: - Web: alphafold.ebi.ac.uk - API - FTP Coverage: - Human - Model organisms - UniProt Example: AF-P04637-F1 โ†’ TP53 predicted structure """) print("\n=== 3. EMDB ===") print(""" EMDB (Electron Microscopy Data Bank): - Cryo-EM maps - 3D reconstructions - EBI Access: - Web: emdb-empiar.org - API Example: EMD-1234 โ†’ Cryo-EM map """) print("\n=== 4. CATH ===") print(""" CATH: - Protein structure classification - Class, Architecture, Topology, Homology - Hierarchical Access: - Web: cathdb.info - API Example: 1.10.8.10 โ†’ Classification """) print("\n=== 5. SCOP ===") print(""" SCOP: - Structural Classification of Proteins - Hierarchical - Manual curation Access: - Web: scop.mrc-lmb.cam.ac.uk - API Example: a.1.1.1 โ†’ Classification """) print("\n=== 6. BMRB ===") print(""" BMRB (Biological Magnetic Resonance Data Bank): - NMR data - Chemical shifts - Metabolomics Access: - Web: bmrb.io - API Example: 15000 โ†’ NMR entry """) print("\n=== 7. Other Structure Databases ===") print(""" 1. PDBe - PDB Europe 2. RCSB PDB - PDB US 3. PDBj - PDB Japan 4. ModBase - Comparative models 5. SWISS-MODEL - Homology models 6. AlphaFold - Predicted """) print("\n=== 8. PDB Format ===") print(""" PDB format: HEADER DNA-BINDING PROTEIN TITLE CRYSTAL STRUCTURE OF P53 COMPND MOL_ID: 1; SOURCE ORGANISM_SCIENTIFIC: HOMO SAPIENS; KEYWDS DNA-BINDING, TUMOR SUPPRESSOR EXPDTA X-RAY DIFFRACTION AUTHOR Y. CHO, S. GORINA, P.D. JEFFREY, N.P. PAVLETICH REVDAT ... ATOM 1 N MET A 1 35.123 45.678 55.789 1.00 50.00 N ... END """) print("\n=== 9. mmCIF Format ===") print(""" mmCIF: - Modern format - PDBx/mmCIF - Larger structures - More metadata Example: _entry.id 1TUP _struct.title 'CRYSTAL STRUCTURE OF P53' _atom_site.group_PDB ATOM _atom_site.id 1 _atom_site.label_atom_id N _atom_site.label_comp_id MET _atom_site.label_asym_id A _atom_site.label_seq_id 1 _atom_site.Cartn_x 35.123 _atom_site.Cartn_y 45.678 _atom_site.Cartn_z 55.789 """) print("\n=== 10. Use Cases ===") print(""" Use cases: - Structure visualization - Docking - Drug design - Structure prediction - Molecular dynamics - Comparative modeling - Function prediction """)

๐Ÿ”— Pathway Databases

๐ŸŽฏ Pathways and Networks

Pathway databases store metabolic, signaling, and regulatory pathways. They are essential for systems biology, drug discovery, and understanding cellular processes.

Pathway Databases

KEGG: Metabolic. Reactome: Curated. WikiPathways: Community. BioCyc: Many. MetaCyc: Metabolic. PANTHER: Classification.

๐Ÿ”— KEGG๐Ÿ”— Reactome๐Ÿ”— WikiPathways๐Ÿ”— BioCyc
# Pathway Databases print("=== 1. KEGG ===") print(""" KEGG (Kyoto Encyclopedia of Genes and Genomes): - Metabolic pathways - Signaling pathways - Disease pathways - Drug pathways Access: - Web: kegg.jp - REST API Databases: - PATHWAY - GENES - COMPOUND - DRUG - DISEASE - BRITE Example: hsa04115 โ†’ p53 signaling pathway """) print("\n=== 2. Reactome ===") print(""" Reactome: - Curated pathways - Human + model organisms - Expert-written - Open source Access: - Web: reactome.org - API Features: - Pathways - Reactions - Proteins - Compounds - Disease Example: R-HSA-69541 โ†’ TP53 regulation """) print("\n=== 3. WikiPathways ===") print(""" WikiPathways: - Community-curated - Open - Many organisms - Wiki-style Access: - Web: wikipathways.org - API Features: - Pathways - Community - Curation - GPML format Example: WP1742 โ†’ p53 pathway """) print("\n=== 4. BioCyc ===") print(""" BioCyc: - Many organisms - Metabolic pathways - Genome databases - Curated Access: - Web: biocyc.org - API Features: - Pathways - Compounds - Genes - Reactions Example: HUMAN โ†’ HumanCyc """) print("\n=== 5. MetaCyc ===") print(""" MetaCyc: - Metabolic pathways - Non-redundant - Curated - Reference Access: - Web: metacyc.org - API Features: - Pathways - Compounds - Enzymes - Reactions Example: PWY-1234 โ†’ Pathway """) print("\n=== 6. PANTHER ===") print(""" PANTHER: - Protein classification - Pathways - Families - Curated Access: - Web: pantherdb.org - API Features: - Pathways - Families - Classification - Ontology Example: P00001 โ†’ Pathway """) print("\n=== 7. Other Pathway Databases ===") print(""" 1. SMPDB - Small molecule pathways - Disease 2. PharmGKB - Pharmacogenomics - Drug pathways 3. INOH - Signaling - Networks 4. NetPath - Signaling - Curated 5. Pathway Commons - Integration - Many sources """) print("\n=== 8. KEGG Format ===") print(""" KEGG format: ENTRY hsa04115 Pathway NAME p53 signaling pathway DESCRIPTION ... CLASS Cellular Processes; Cell growth and death PATHWAY_MAP hsa04115 GENE 7157 TP53 ... COMPOUND C00002 ATP ... REFERENCE ... // KGML (KEGG Markup Language): ... """) print("\n=== 9. Use Cases ===") print(""" Use cases: - Pathway analysis - Enrichment analysis - Systems biology - Drug discovery - Disease mechanisms - Metabolic engineering - Network analysis """) print("\n=== 10. Cross-References ===") print(""" Cross-references: - KEGG โ†’ UniProt - KEGG โ†’ Ensembl - KEGG โ†’ PDB - Reactome โ†’ UniProt - Reactome โ†’ Ensembl - WikiPathways โ†’ Ensembl Example: KEGG hsa04115 โ†’ UniProt P04637 โ†’ Ensembl ENSG00000141510 """)

๐Ÿ“ˆ Expression Databases

๐ŸŽฏ Gene Expression Data

Expression databases store gene expression data from microarrays, RNA-seq, and proteomics. They are essential for transcriptomics, functional genomics, and disease research.

Expression Databases

GEO: NCBI. ArrayExpress: EBI. GTEx: Human tissues. TCGA: Cancer. HPA: Protein. CCLE: Cell lines.

๐Ÿ“ˆ GEO๐Ÿ“ˆ ArrayExpress๐Ÿ“ˆ GTEx๐Ÿ“ˆ TCGA
# Expression Databases print("=== 1. GEO ===") print(""" GEO (Gene Expression Omnibus): - NCBI - Microarray + RNA-seq - > 5 million samples - Free Access: - Web: ncbi.nlm.nih.gov/geo - Entrez - FTP Features: - Datasets - Samples - Platforms - Series Example: GSE12345 โ†’ Expression dataset """) print("\n=== 2. ArrayExpress ===") print(""" ArrayExpress: - EBI - Microarray + RNA-seq - Curated - Free Access: - Web: ebi.ac.uk/arrayexpress - API Features: - Experiments - Samples - Protocols - Data Example: E-MTAB-1234 โ†’ Experiment """) print("\n=== 3. GTEx ===") print(""" GTEx (Genotype-Tissue Expression): - Human tissues - Expression + genotype - Multiple tissues - Free Access: - Web: gtexportal.org - API Features: - Expression - eQTLs - Tissues - Genotypes Example: GTEX-1234 โ†’ Sample """) print("\n=== 4. TCGA ===") print(""" TCGA (The Cancer Genome Atlas): - Cancer - Multi-omics - Many cancer types - Free Access: - Web: cancer.gov/tcga - GDC Features: - Expression - Mutations - Copy number - Methylation Example: TCGA-01-1234 โ†’ Sample """) print("\n=== 5. Human Protein Atlas ===") print(""" Human Protein Atlas: - Protein expression - Localization - Tissue - Cell Access: - Web: proteinatlas.org - API Features: - Expression - Localization - Tissue - Cancer Example: TP53 โ†’ Protein expression """) print("\n=== 6. CCLE ===") print(""" CCLE (Cancer Cell Line Encyclopedia): - Cancer cell lines - Expression - Mutations - Drug response Access: - Web: broadinstitute.org/ccle - API Features: - Expression - Mutations - Copy number - Drug Example: A549 โ†’ Cell line """) print("\n=== 7. Other Expression Databases ===") print(""" 1. ENCODE - Functional genomics - Epigenomics 2. Roadmap Epigenomics - Epigenomes - Human 3. FANTOM - Transcriptomics - Mouse/human 4. Single Cell Portal - Single-cell - Broad 5. Human Cell Atlas - Single-cell - Human """) print("\n=== 8. GEO Format ===") print(""" GEO: Series: GSE12345 Samples: GSM123456, GSM123457, ... Platform: GPL12345 Sample: GSM123456 Title: Control sample Organism: Homo sapiens Tissue: Liver Treatment: None Data: Raw: .CEL, .fastq Processed: .txt, .csv """) print("\n=== 9. Use Cases ===") print(""" Use cases: - Differential expression - Co-expression - Biomarker discovery - Drug response - Disease mechanisms - Tissue specificity - Single-cell """) print("\n=== 10. Analysis ===") print(""" Analysis: - Differential expression - Clustering - PCA - Pathway enrichment - Network analysis - Machine learning Tools: - GEO2R - R (limma, DESeq2) - Python (scikit-learn) """)

๐Ÿ› ๏ธ Tools (Entrez, UniProt API)

๐ŸŽฏ Accessing Databases

Most biological databases offer APIs and tools for programmatic access. Here are the most important ones.

Access Tools

Entrez: NCBI. UniProt API: REST. Ensembl API: REST. PDB API: REST. KEGG API: REST. Biopython: Python.

๐Ÿ› ๏ธ Entrez๐Ÿ› ๏ธ UniProt API๐Ÿ› ๏ธ Ensembl API๐Ÿ› ๏ธ Biopython
# Tools print("=== 1. NCBI Entrez (E-utilities) ===") print(""" Entrez: - NCBI API - Access to all NCBI databases - E-utilities Databases: - nucleotide - protein - pubmed - gene - genome - structure E-utilities: - esearch: search - efetch: retrieve - esummary: summary - elink: cross-reference URL: https://eutils.ncbi.nlm.nih.gov/entrez/eutils/ Example: esearch.fcgi?db=protein&term=TP53&retmax=10 """) print("\n=== 2. UniProt REST API ===") print(""" UniProt API: - RESTful - Protein data - Search, retrieve, map Endpoints: - /uniprotkb/search - /uniprotkb/{accession} - /uniprotkb/stream - /idmapping/run URL: https://rest.uniprot.org/ Example: https://rest.uniprot.org/uniprotkb/P04637 """) print("\n=== 3. Ensembl REST API ===") print(""" Ensembl API: - RESTful - Genome data - Genes, variants, sequences Endpoints: - /lookup/id/{id} - /sequence/id/{id} - /vep/{species}/hgvs/{hgvs} - /overlap/region/{species}/{region} URL: https://rest.ensembl.org/ Example: https://rest.ensembl.org/lookup/id/ENSG00000141510 """) print("\n=== 4. PDB API ===") print(""" PDB API: - RESTful - Structure data - Search, retrieve Endpoints: - /rest/v1/entry/{id} - /rest/v1/search - /rest/v1/core/{id} URL: https://data.rcsb.org/ Example: https://data.rcsb.org/rest/v1/core/entry/1TUP """) print("\n=== 5. KEGG API ===") print(""" KEGG API: - RESTful - Pathway data - Genes, compounds Endpoints: - /get/{db}/{entry} - /list/{db} - /find/{db}/{query} URL: https://rest.kegg.jp/ Example: https://rest.kegg.jp/get/hsa04115 """) print("\n=== 6. Biopython ===") print(""" Biopython: - Python library - Entrez - Parsing - Analysis Modules: - Bio.Entrez - Bio.SeqIO - Bio.PDB - Bio.Blast Example: from Bio import Entrez Entrez.email = "your@email.com" handle = Entrez.esearch(db="protein", term="TP53") record = Entrez.read(handle) """) print("\n=== 7. Other Tools ===") print(""" 1. Requests - Python HTTP - REST APIs 2. curl - Command line - HTTP 3. wget - Download - FTP 4. Entrez Direct - Command line - NCBI 5. Dataset - NCBI - Command line """) print("\n=== 8. Rate Limits ===") print(""" Rate limits: - NCBI: 3 requests/second (without API key) - NCBI: 10 requests/second (with API key) - UniProt: reasonable - Ensembl: 15 requests/second - PDB: reasonable - KEGG: reasonable Best practices: - Use API key - Cache results - Respect limits - Batch requests """) print("\n=== 9. Example Queries ===") print(""" NCBI Entrez: esearch.fcgi?db=protein&term=TP53[gene]+AND+human[orgn] efetch.fcgi?db=protein&id=NP_000537&rettype=fasta UniProt: https://rest.uniprot.org/uniprotkb/search?query=TP53+AND+organism_id:9606 Ensembl: https://rest.ensembl.org/lookup/id/ENSG00000141510?expand=1 PDB: https://data.rcsb.org/rest/v1/core/entry/1TUP KEGG: https://rest.kegg.jp/get/hsa04115 """) print("\n=== 10. Best Practices ===") print(""" 1. Use API key 2. Cache results 3. Respect rate limits 4. Handle errors 5. Use HTTPS 6. Read documentation 7. Cite databases 8. Version control """)

๐Ÿ Python Integration

๐ŸŽฏ Accessing Databases from Python

Python is the language of choice for bioinformatics. Here's how to access biological databases from Python.

Python

Biopython: Entrez. Requests: REST. Pandas: Data. Matplotlib: Plot. Jupyter: Notebook. Automate: Pipeline.

๐Ÿ Biopython๐Ÿ Requests๐Ÿ Pandas๐Ÿ Automate
# Python Integration print("=== 1. NCBI Entrez with Biopython ===") print(""" from Bio import Entrez # Set email (required) Entrez.email = "your.email@example.com" # Search handle = Entrez.esearch(db="protein", term="TP53 AND human", retmax=10) record = Entrez.read(handle) handle.close() print(record["IdList"]) # Fetch handle = Entrez.efetch(db="protein", id=record["IdList"], rettype="fasta") sequences = handle.read() handle.close() """) print("\n=== 2. UniProt REST API ===") print(""" import requests def get_uniprot(accession): url = f"https://rest.uniprot.org/uniprotkb/{accession}.json" response = requests.get(url) if response.status_code == 200: return response.json() return None # Example data = get_uniprot("P04637") if data: print(data["primaryAccession"]) print(data["proteinDescription"]["recommendedName"]["fullName"]["value"]) """) print("\n=== 3. Ensembl REST API ===") print(""" import requests def get_ensembl_gene(gene_id): url = f"https://rest.ensembl.org/lookup/id/{gene_id}?expand=1" headers = {"Content-Type": "application/json"} response = requests.get(url, headers=headers) if response.status_code == 200: return response.json() return None # Example gene = get_ensembl_gene("ENSG00000141510") if gene: print(gene["display_name"]) print(gene["description"]) """) print("\n=== 4. PDB API ===") print(""" import requests def get_pdb_entry(pdb_id): url = f"https://data.rcsb.org/rest/v1/core/entry/{pdb_id}" response = requests.get(url) if response.status_code == 200: return response.json() return None # Example entry = get_pdb_entry("1TUP") if entry: print(entry["struct"]["title"]) """) print("\n=== 5. KEGG API ===") print(""" import requests def get_kegg_pathway(pathway_id): url = f"https://rest.kegg.jp/get/{pathway_id}" response = requests.get(url) if response.status_code == 200: return response.text return None # Example pathway = get_kegg_pathway("hsa04115") if pathway: print(pathway[:500]) """) print("\n=== 6. Parse FASTA ===") print(""" from Bio import SeqIO def parse_fasta(filepath): sequences = [] for record in SeqIO.parse(filepath, "fasta"): sequences.append({ "id": record.id, "description": record.description, "sequence": str(record.seq) }) return sequences # Example # sequences = parse_fasta("sequences.fasta") """) print("\n=== 7. Parse GenBank ===") print(""" from Bio import SeqIO def parse_genbank(filepath): records = [] for record in SeqIO.parse(filepath, "genbank"): records.append({ "id": record.id, "name": record.name, "description": record.description, "sequence": str(record.seq), "features": [(f.type, f.location) for f in record.features] }) return records # Example # records = parse_genbank("sequence.gb") """) print("\n=== 8. Parse PDB ===") print(""" from Bio.PDB import PDBParser def parse_pdb(filepath): parser = PDBParser() structure = parser.get_structure("protein", filepath) for model in structure: for chain in model: for residue in chain: print(residue) return structure # Example # structure = parse_pdb("1tup.pdb") """) print("\n=== 9. Complete Pipeline ===") print(""" from Bio import Entrez, SeqIO import requests import pandas as pd def database_pipeline(gene): \"\"\"Complete database pipeline.\"\"\" # Search NCBI Entrez.email = "your.email@example.com" handle = Entrez.esearch(db="protein", term=f"{gene} AND human", retmax=5) record = Entrez.read(handle) handle.close() # Fetch handle = Entrez.efetch(db="protein", id=record["IdList"], rettype="fasta") sequences = list(SeqIO.parse(handle, "fasta")) handle.close() # Get UniProt uniprot = get_uniprot("P04637") # Get Ensembl ensembl = get_ensembl_gene("ENSG00000141510") # Combine data = { "gene": gene, "ncbi_count": len(sequences), "uniprot": uniprot["primaryAccession"] if uniprot else None, "ensembl": ensembl["display_name"] if ensembl else None } return data # Use # data = database_pipeline("TP53") # print(data) """) print("\n=== 10. Visualization ===") print(""" import matplotlib.pyplot as plt import pandas as pd def plot_expression(data): \"\"\"Plot expression data.\"\"\" df = pd.DataFrame(data) plt.figure(figsize=(10, 6)) plt.bar(df["tissue"], df["expression"], color='purple') plt.xlabel("Tissue") plt.ylabel("Expression") plt.title("Gene Expression") plt.xticks(rotation=45) plt.tight_layout() plt.savefig("expression.png") plt.show() # Use # plot_expression(data) """)

โš–๏ธ Database Comparison

๐ŸŽฏ Which Database?

Choosing the right database depends on your data type and analysis goals.

Comparison

NCBI: Comprehensive. UniProt: Proteins. Ensembl: Genomes. PDB: Structures. KEGG: Pathways. GEO: Expression.

โš–๏ธ NCBIโš–๏ธ UniProtโš–๏ธ Ensemblโš–๏ธ PDB
Database Type Data Access Best For
NCBICompositeSequencesWeb, APIGeneral
UniProtSecondaryProteinsWeb, APIProtein sequences
EnsemblCompositeGenomesWeb, APIGenomes
PDBPrimaryStructuresWeb, API3D structures
KEGGSecondaryPathwaysWeb, APIPathways
ReactomeSecondaryPathwaysWeb, APICurated pathways
GEOPrimaryExpressionWeb, APIExpression
PfamSecondaryFamiliesWeb, APIProtein families
InterProCompositeDomainsWeb, APIDomains
AlphaFoldSecondaryStructuresWeb, APIPredicted
# Decision Guide print("=== Choose Database by Data Type ===") print(""" DNA sequences: โ†’ GenBank โ†’ RefSeq โ†’ Ensembl Protein sequences: โ†’ UniProt โ†’ RefSeq Protein structures: โ†’ PDB โ†’ AlphaFold DB Protein families: โ†’ Pfam โ†’ InterPro Genomes: โ†’ Ensembl โ†’ UCSC โ†’ NCBI Genome Pathways: โ†’ KEGG โ†’ Reactome โ†’ WikiPathways Expression: โ†’ GEO โ†’ ArrayExpress โ†’ GTEx Interactions: โ†’ STRING โ†’ BioGRID Diseases: โ†’ OMIM โ†’ ClinVar """) print("\n=== Speed Comparison ===") print(f"{'Database':<20} {'Size':<20} {'Access':<15} {'API':<10}") print("-" * 65) print(f"{'NCBI':<20} {'3B+ sequences':<20} {'Web, API':<15} {'Yes':<10}") print(f"{'UniProt':<20} {'200M+ proteins':<20} {'Web, API':<15} {'Yes':<10}") print(f"{'Ensembl':<20} {'100+ genomes':<20} {'Web, API':<15} {'Yes':<10}") print(f"{'PDB':<20} {'200K+ structures':<20} {'Web, API':<15} {'Yes':<10}") print(f"{'KEGG':<20} {'500+ pathways':<20} {'Web, API':<15} {'Yes':<10}") print("\n=== When to Use Which ===") print(""" USE NCBI: โœ“ General โœ“ Sequences โœ“ Literature โœ“ BLAST USE UniProt: โœ“ Proteins โœ“ Functional annotation โœ“ Cross-references USE Ensembl: โœ“ Genomes โœ“ Comparative โœ“ Variants USE PDB: โœ“ Structures โœ“ Drug design โœ“ Visualization USE KEGG: โœ“ Pathways โœ“ Metabolic โœ“ Disease USE GEO: โœ“ Expression โœ“ Differential โœ“ Biomarkers """)

โ“ FAQ

๐ŸŽฏ Frequently Asked Questions

Answers to common questions about biological databases.

What is a biological database?

A biological database is an organized collection of biological data (sequences, structures, pathways, expression) that is stored, managed, and made accessible for querying and analysis. Examples include NCBI, UniProt, Ensembl, PDB, and KEGG.

What is the difference between primary and secondary databases?

Primary databases store raw experimental data (e.g., GenBank, PDB, GEO). Secondary databases store derived, curated, or integrated data (e.g., UniProt, Pfam, KEGG, Reactome). Primary data is submitted by researchers; secondary data is curated and annotated.

Which biological database is best for protein sequences?

UniProt is the most comprehensive protein sequence database, with Swiss-Prot (curated) and TrEMBL (unreviewed). For protein structures, use PDB. For protein families and domains, use Pfam and InterPro.

How do I access biological databases programmatically?

Most databases offer APIs: NCBI Entrez (E-utilities), UniProt REST API, Ensembl REST API, PDB API, and KEGG REST API. You can access them from Python using Biopython (Entrez), requests, or dedicated libraries.

What is the difference between GenBank and RefSeq?

GenBank is a primary database containing all submitted sequences (may include redundancy and errors). RefSeq is a secondary, curated database of reference sequences for genomes, transcripts, and proteins, with non-redundant, well-annotated entries.

What is an accession number?

An accession number is a unique identifier assigned to a sequence, structure, or other biological record in a database. It is stable and versioned. Examples: NC_000001.11, NM_000546.6, P04637.

What is the difference between KEGG and Reactome?

KEGG is a comprehensive database of metabolic and signaling pathways, with a focus on metabolism and disease. Reactome is a curated, expert-written database of human and model organism pathways, with detailed molecular reactions.

How do I download data from biological databases?

Most databases offer FTP downloads, API access, and web interfaces. Use FTP for bulk downloads, APIs for programmatic access, and web for browsing. Examples: NCBI FTP, UniProt FTP, Ensembl FTP.

What is the best database for gene expression data?

GEO (Gene Expression Omnibus) is the largest public repository for gene expression data (microarray, RNA-seq). For human tissue-specific expression, use GTEx. For cancer, use TCGA. For protein expression, use Human Protein Atlas.

How do I cite a biological database?

Always cite the database and its primary publication. For example: "UniProt Consortium (2023) UniProt: the Universal Protein Knowledgebase. Nucleic Acids Research." Check each database's website for citation guidelines.

# Quick Reference print("=== Quick Answers ===") print(""" Q: What is a biological database? A: Organized collection of biological data. Q: Primary vs secondary? A: Primary = raw; secondary = curated. Q: Best for protein? A: UniProt. Q: Best for structures? A: PDB, AlphaFold. Q: Best for genomes? A: Ensembl, UCSC. Q: Best for pathways? A: KEGG, Reactome. Q: Best for expression? A: GEO, GTEx. Q: How to access? A: Web, API, FTP, Biopython. Q: GenBank vs RefSeq? A: GenBank = primary; RefSeq = curated. Q: Accession number? A: Unique identifier. Q: How to cite? A: Cite database and publication. Q: Free? A: Yes โ€” all major databases are free. """)

๐Ÿš€ Future

๐ŸŽฏ Where Databases Are Going

The future is AI-integrated, knowledge graphs, cloud-native, and multi-omics.

Future Directions

AI: Integration. Graphs: Knowledge. Cloud: Native. Multi-omics: Integrated. Real-time: Updates. FAIR: Data principles.

๐Ÿš€ AI๐Ÿš€ Knowledge Graphs๐Ÿš€ Cloud๐Ÿš€ Multi-omics
# Future of Biological Databases print("=== 1. AI Integration ===") print(""" Trend: AI + databases Ideas: - AI-powered search - Intelligent annotation - Predictive models - Knowledge discovery Examples: - AlphaFold DB - GPT for biology - AI curation Future: - Smart databases - Automated annotation - Predictive """) print("\n=== 2. Knowledge Graphs ===") print(""" Trend: Knowledge graphs Current: - Relational databases - Flat files Future: - Knowledge graphs - Linked data - Semantic web Examples: - Bio2RDF - Wikidata - SPOKE - Hetionet Future: - Integrated knowledge - Reasoning - Discovery """) print("\n=== 3. Cloud-Native ===") print(""" Trend: Cloud databases Current: - Local servers - FTP downloads Future: - Cloud-native - Serverless - Auto-scaling - Pay per use Examples: - AWS Open Data - Google Cloud - Azure - Terra Future: - Petabyte-scale - Real-time - Global """) print("\n=== 4. Multi-Omics ===") print(""" Trend: Multi-omics integration Current: - Separate databases - Genomics, proteomics, etc. Future: - Integrated - Multi-omics - Systems biology Examples: - TCGA - GTEx - Human Cell Atlas - UK Biobank Future: - Unified - Integrated - Systems-level """) print("\n=== 5. Real-Time Data ===") print(""" Trend: Real-time updates Current: - Periodic releases - Static Future: - Real-time - Streaming - Live Examples: - COVID-19 databases - GISAID - Nextstrain Future: - Real-time - Continuous - Dynamic """) print("\n=== 6. FAIR Principles ===") print(""" Trend: FAIR data FAIR: - Findable - Accessible - Interoperable - Reusable Future: - All databases FAIR - Standardized - Machine-readable Examples: - DataCite - DOIs - Metadata standards """) # TIMELINE print("\n=== Timeline ===") print(""" Past: 1970s: PDB, GenBank 1980s: EMBL, Swiss-Prot 1990s: NCBI, Entrez, KEGG 2000s: Ensembl, UniProt 2010s: Big data, APIs 2020s: AI, knowledge graphs Present: 2025: Cloud-native 2026: AI-integrated Future: 2030: Knowledge graphs 2035: Multi-omics 2040: Real-time? """) print("\n=== Final Thoughts ===") print(""" Biological databases are: - Essential for research - Growing exponentially - Becoming more integrated - AI-powered - Cloud-native The future is: - AI-integrated - Knowledge graphs - Cloud-native - Multi-omics - Real-time - FAIR "Data is the new oil โ€” but biological data is the new gold." Keep exploring. Keep building. """)