Capturing, Selecting, Managing & Exchanging Data

Overview

In computer science, handling data efficiently and accurately is essential for database management and data-driven applications. This note covers four key areas: capturing data, selecting data, managing data, and exchanging data. Each area includes various methods and techniques for processing and transferring data, helping users work effectively with large datasets and make informed decisions.

Capturing Data

Data capturing refers to collecting data from various sources to use within a system. Methods include:

• Forms: Commonly used to collect structured information from users (e.g., online forms or paper forms). Forms typically include fields like name, date, and email to ensure consistent data input.
• Optical Character Recognition (OCR): Software that converts scanned images of text into editable digital text. Useful for digitising paper documents but may struggle with poor-quality images or handwriting.
• Optical Mark Recognition (OMR): Reads marks made on forms, such as multiple-choice answer sheets, where the user fills in bubbles. This method is efficient for processing surveys or exams but is limited to specific formats.
• Sensors: Capture data from the physical environment, such as temperature, motion, or light sensors. Widely used in IoT (Internet of Things) applications to automatically gather real-time data.
• Data Mining: Extracting patterns and insights from large datasets, often using statistical or machine learning algorithms. Data mining is commonly used for trend analysis, customer segmentation, and predictive analytics.

Example

An online registration form for a conference might capture:

• Text Fields: Name, Email, Organisation.
• Drop-down Lists: Country, Job Title.
• Checkboxes: Sessions the attendee wishes to join.

Selecting Data

Data selection involves retrieving specific data from a dataset or database. Methods include:

• Query By Example (QBE): A user-friendly approach to building queries visually, where users select fields, tables, criteria, and sorting options. Commonly used in database management tools like Microsoft Access.
• SQL (Structured Query Language): A programming language used to manage and query data in relational databases. SQL allows more flexibility and complexity in selecting data than QBE.
• Boolean Expressions (AND, OR, NOT): Logical operators used in queries to refine search results. For example, WHERE Age > 18 AND City = 'London' retrieves records for adults in London.

Example SQL Query

SELECT * FROM Customers
WHERE City = 'New York' AND Age > 25;

The Purpose of Indexing

• Indexing: An optimisation technique that creates a data structure to speed up searches. It works similarly to an index in a book, allowing faster access to rows in a table based on indexed columns.
• Benefits of Indexing: Faster querying and retrieval of data, especially in large datasets. However, it can increase storage requirements and slow down data modifications (inserts, updates, deletes).

Managing Data

Data management involves editing and processing data within a database to keep it accurate, current, and usable. Key tasks include:

• Adding Data: Inserting new records, e.g., adding a new customer to a database.
• Editing Data: Modifying existing records, such as updating a customer's contact information.
• Deleting Data: Removing records, like deleting outdated product listings.
• Data Manipulation:
  • Arithmetic Functions: Performing calculations directly in a query. For example, calculating total price by multiplying Quantity and Unit_Price.
  • Concatenation: Joining strings, like combining first and last names.
  • String and Date Functions: E.g., extracting a substring or calculating the difference between two dates.

Example SQL Command

UPDATE Customers
SET Address = '123 New Street'
WHERE CustomerID = 101;

Exchanging Data

Data exchange refers to transferring data between systems, users, or locations. This includes choosing the right format and medium for transferring data effectively.

Formats for Data Exchange:

• CSV (Comma-Separated Values): Plain text format where data fields are separated by commas. Widely supported and easy to parse, but lacks data validation and structure beyond rows and columns.
• JSON (JavaScript Object Notation): A lightweight, human-readable format for structuring data as key-value pairs, commonly used in web APIs.
• XML (Extensible Markup Language): A structured format that uses tags to organise data. Often used for exchanging data in complex, hierarchical structures.

Data Transfer Methods:

• Electronic: Using email, file-sharing services, or cloud storage to transfer digital data (e.g., attaching a CSV file to an email).
• Non-Electronic: Transferring data on physical media like printed reports or USB drives.

Communication Mediums:

• Email: Efficient for small data files but may not be suitable for sensitive or large datasets.
• API (Application Programming Interface): Allows different software systems to communicate, often using JSON or XML for data transfer.
• Physical Mediums: Printed reports or storage devices like USB sticks, useful when internet access is unavailable or for secure offline access.

Example of JSON Format

{
  "CustomerID": 101,
  "Name": "Alice Smith",
  "City": "New York",
  "Age": 30
}

Note Summary