Oracle Data Pump is a newer, faster and more flexible alternative to
the "exp" and "imp" utilities used in previous Oracle versions. In
addition to basic import and export functionality data pump provides a
PL/SQL API and support for external tables.
- Getting Started
- Table Exports/Imports
- Schema Exports/Imports
- Database Exports/Imports
- INCLUDE and EXCLUDE
- Network Exports/Imports (NETWORK_LINK)
- Miscellaneous Information
- Data Pump API
- External Tables
- Help
Getting Started
For the examples to work we must first unlock the SCOTT account and
create a directory object it can access. The directory object is only a
pointer to a physical directory, creating it does not actually create
the physical directory on the file system.
CONN / AS SYSDBA ALTER USER scott IDENTIFIED BY tiger ACCOUNT UNLOCK; CREATE OR REPLACE DIRECTORY test_dir AS '/u01/app/oracle/oradata/'; GRANT READ, WRITE ON DIRECTORY test_dir TO scott;
Existing directories can be queried using the
ALL_DIRECTORIES view.Table Exports/Imports
The
TABLES parameter is used to specify the tables that
are to be exported. The following is an example of the table export and
import syntax.expdp scott/tiger@db10g tables=EMP,DEPT directory=TEST_DIR
dumpfile=EMP_DEPT.dmp logfile=expdpEMP_DEPT.log impdp scott/tiger@db10g tables=EMP,DEPT directory=TEST_DIR
dumpfile=EMP_DEPT.dmp logfile=impdpEMP_DEPT.log
For example output files see expdpEMP_DEPT.log and impdpEMP_DEPT.log.
The
TABLE_EXISTS_ACTION=APPEND parameter allows data to be imported into existing tables.Schema Exports/Imports
The
OWNER parameter of exp has been replaced by the SCHEMAS
parameter which is used to specify the schemas to be exported. The
following is an example of the schema export and import syntax.expdp scott/tiger@db10g schemas=SCOTT directory=TEST_DIR
dumpfile=SCOTT.dmp logfile=expdpSCOTT.log impdp scott/tiger@db10g schemas=SCOTT directory=TEST_DIR
dumpfile=SCOTT.dmp logfile=impdpSCOTT.log
For example output files see expdpSCOTT.log and impdpSCOTT.log.
Database Exports/Imports
The
FULL parameter indicates that a complete database export is required. The following is an example of the
full database export and import syntax.expdp system/password@db10g full=Y directory=TEST_DIR
dumpfile=DB10G.dmp logfile=expdpDB10G.log impdp system/password@db10g full=Y directory=TEST_DIR
dumpfile=DB10G.dmp logfile=impdpDB10G.log
For an example output file see expdpDB10G.log.
INCLUDE and EXCLUDE
The
INCLUDE and EXCLUDE parameters can be used to limit the export/import to specific objects. When the INCLUDE parameter is used, only those objects specified by it will be included in the export/import. When the EXCLUDE
parameter is used, all objects except those specified by it will be
included in the export/import. The two parameters are mutually
exclusive, so use the parameter that requires the least entries to give
you the result you require. The basic syntax for both parameters is the
same.INCLUDE=object_type[:name_clause] [, ...] EXCLUDE=object_type[:name_clause] [, ...]
The following code shows how they can be used as command line parameters.
expdp scott/tiger@db10g schemas=SCOTT include=TABLE:"IN ('EMP', 'DEPT')"
directory=TEST_DIR dumpfile=SCOTT.dmp logfile=expdpSCOTT.log expdp scott/tiger@db10g schemas=SCOTT exclude=TABLE:"= 'BONUS'"
directory=TEST_DIR dumpfile=SCOTT.dmp logfile=expdpSCOTT.log
If the parameter is used from the command line, depending on your OS,
the special characters in the clause may need to be escaped, as
follows. Because of this, it is easier to use a parameter file.
include=TABLE:\"IN (\'EMP\', \'DEPT\')\"
A single import/export can include multiple references to the
parameters, so to export tables, views and some packages we could use
either of the following approaches.
INCLUDE=TABLE,VIEW,PACKAGE:"LIKE '%API'" or INCLUDE=TABLE INCLUDE=VIEW INCLUDE=PACKAGE:"LIKE '%API'"
The valid object type paths that can be included or excluded can be displayed using the
DATABASE_EXPORT_OBJECTS, SCHEMA_EXPORT_OBJECTS, and TABLE_EXPORT_OBJECTS views.Network Exports/Imports (NETWORK_LINK)
The
NETWORK_LINK parameter identifies a database link to
be used as the source for a network export/import. The following
database link will be used to demonstrate its use.CONN / AS SYSDBA GRANT CREATE DATABASE LINK TO test; CONN test/test CREATE DATABASE LINK remote_scott CONNECT TO scott IDENTIFIED BY tiger
USING 'DEV';
In the case of exports, the
NETWORK_LINK parameter
identifies the database link pointing to the source server. The objects
are exported from the source server in the normal manner, but written to
a directory object on the local server, rather than one on the source
server. Both the local and remote users require the EXP_FULL_DATABASE role granted to them.expdp test/test@db10g tables=SCOTT.EMP network_link=REMOTE_SCOTT
directory=TEST_DIR dumpfile=EMP.dmp logfile=expdpEMP.log
For imports, the
NETWORK_LINK parameter also identifies
the database link pointing to the source server. The difference here is
the objects are imported directly from the source into the local server
without being written to a dump file. Although there is no need for a DUMPFILE
parameter, a directory object is still required for the logs associated
with the operation. Both the local and remote users require the IMP_FULL_DATABASE role granted to them.impdp test/test@db10g tables=SCOTT.EMP network_link=REMOTE_SCOTT
directory=TEST_DIR logfile=impdpSCOTT.log remap_schema=SCOTT:TEST
Miscellaneous Information
Unlike the original exp and imp utilities all data pump ".dmp" and
".log" files are created on the Oracle server, not the client machine.
All data pump actions are performed by multiple jobs (server
processes not DBMS_JOB jobs). These jobs are controlled by a
master control process which uses Advanced Queuing. At runtime an
advanced queue table, named after the job name, is created and used by
the
master control process. The table is dropped on completion of the data
pump job. The job and the advanced queue can be named
using the
JOB_NAME parameter. Cancelling the client process
does not stop the associated data pump job. Issuing "ctrl+c"
on the client during a job stops the client output and presents a
command prompt. Typing "status" at this prompt allows you to monitor the
current job.
Export> status
Job: SYS_EXPORT_FULL_01
Operation: EXPORT
Mode: FULL
State: EXECUTING
Bytes Processed: 0
Current Parallelism: 1
Job Error Count: 0
Dump File: D:\TEMP\DB10G.DMP
bytes written: 4,096
Worker 1 Status:
State: EXECUTING
Object Schema: SYSMAN
Object Name: MGMT_CONTAINER_CRED_ARRAY
Object Type: DATABASE_EXPORT/SCHEMA/TYPE/TYPE_SPEC
Completed Objects: 261
Total Objects: 261
Data pump performance can be improved by using the
PARALLEL parameter. This should be used in conjunction with the "%U" wildcard in the DUMPFILE parameter to allow multiple dumpfiles to be created or read.expdp scott/tiger@db10g schemas=SCOTT directory=TEST_DIR parallel=4
dumpfile=SCOTT_%U.dmp logfile=expdpSCOTT.log
The
DBA_DATAPUMP_JOBS view can be used to monitor the current jobs.system@db10g> select * from dba_datapump_jobs; OWNER_NAME JOB_NAME OPERATION ------------------------------ ------------------------------ ------------------------------ JOB_MODE STATE DEGREE ATTACHED_SESSIONS ------------------------------ ------------------------------ ---------- ----------------- SYSTEM SYS_EXPORT_FULL_01 EXPORT FULL EXECUTING 1 1
Data Pump API
Along with the data pump utilities Oracle provide an PL/SQL API. The
following is an example of how this API can be used to perform a schema
export.
SET SERVEROUTPUT ON SIZE 1000000
DECLARE
l_dp_handle NUMBER;
l_last_job_state VARCHAR2(30) := 'UNDEFINED';
l_job_state VARCHAR2(30) := 'UNDEFINED';
l_sts KU$_STATUS;
BEGIN
l_dp_handle := DBMS_DATAPUMP.open(
operation => 'EXPORT',
job_mode => 'SCHEMA',
remote_link => NULL,
job_name => 'EMP_EXPORT',
version => 'LATEST');
DBMS_DATAPUMP.add_file(
handle => l_dp_handle,
filename => 'SCOTT.dmp',
directory => 'TEST_DIR');
DBMS_DATAPUMP.add_file(
handle => l_dp_handle,
filename => 'SCOTT.log',
directory => 'TEST_DIR',
filetype => DBMS_DATAPUMP.KU$_FILE_TYPE_LOG_FILE);
DBMS_DATAPUMP.metadata_filter(
handle => l_dp_handle,
name => 'SCHEMA_EXPR',
value => '= ''SCOTT''');
DBMS_DATAPUMP.start_job(l_dp_handle);
DBMS_DATAPUMP.detach(l_dp_handle);
END;
/
Once the job has started the status can be checked using.
system@db10g> select * from dba_datapump_jobs;
External Tables
Oracle have incorporated support for data pump technology into
external tables. The ORACLE_DATAPUMP access driver can be used to unload
data to data pump export files and subsequently reload it. The unload of
data occurs when the external table is created using the "AS" clause.
CREATE TABLE emp_xt
ORGANIZATION EXTERNAL
(
TYPE ORACLE_DATAPUMP
DEFAULT DIRECTORY test_dir
LOCATION ('emp_xt.dmp')
)
AS SELECT * FROM emp;
The data can then be queried using the following.
SELECT * FROM emp_xt;
The syntax to create the external table pointing to an existing file is similar, but without the "AS" clause.
DROP TABLE emp_xt;
CREATE TABLE emp_xt (
EMPNO NUMBER(4),
ENAME VARCHAR2(10),
JOB VARCHAR2(9),
MGR NUMBER(4),
HIREDATE DATE,
SAL NUMBER(7,2),
COMM NUMBER(7,2),
DEPTNO NUMBER(2))
ORGANIZATION EXTERNAL (
TYPE ORACLE_DATAPUMP
DEFAULT DIRECTORY test_dir
LOCATION ('emp_xt.dmp')
);
SELECT * FROM emp_xt;
Help
The
HELP=Y option displays the available parameters.expdp
expdp help=y Export: Release 10.1.0.2.0 - Production on Tuesday, 23 March, 2004 8:33 Copyright (c) 2003, Oracle. All rights reserved. The Data Pump export utility provides a mechanism for transferring data
objects
between Oracle databases. The utility is invoked with the following command:
Example: expdp scott/tiger DIRECTORY=dmpdir DUMPFILE=scott.dmp
You can control how Export runs by entering the 'expdp' command followed
by various parameters. To specify parameters, you use keywords:
Format: expdp KEYWORD=value or KEYWORD=(value1,value2,...,valueN)
Example: expdp scott/tiger DUMPFILE=scott.dmp DIRECTORY=dmpdir SCHEMAS=scott
or TABLES=(T1:P1,T1:P2), if T1 is partitioned table
USERID must be the first parameter on the command line.
Keyword Description (Default)
------------------------------------------------------------------------------
ATTACH Attach to existing job, e.g. ATTACH [=job name].
CONTENT Specifies data to unload where the valid keywords are:
(ALL), DATA_ONLY, and METADATA_ONLY.
DIRECTORY Directory object to be used for dumpfiles and logfiles.
DUMPFILE List of destination dump files (expdat.dmp),
e.g. DUMPFILE=scott1.dmp, scott2.dmp, dmpdir:scott3.dmp.
ESTIMATE Calculate job estimates where the valid keywords are:
(BLOCKS) and STATISTICS.
ESTIMATE_ONLY Calculate job estimates without performing the export.
EXCLUDE Exclude specific object types, e.g. EXCLUDE=TABLE:EMP.
FILESIZE Specify the size of each dumpfile in units of bytes.
FLASHBACK_SCN SCN used to set session snapshot back to.
FLASHBACK_TIME Time used to get the SCN closest to the specified time.
FULL Export entire database (N).
HELP Display Help messages (N).
INCLUDE Include specific object types, e.g. INCLUDE=TABLE_DATA.
JOB_NAME Name of export job to create.
LOGFILE Log file name (export.log).
NETWORK_LINK Name of remote database link to the source system.
NOLOGFILE Do not write logfile (N).
PARALLEL Change the number of active workers for current job.
PARFILE Specify parameter file.
QUERY Predicate clause used to export a subset of a table.
SCHEMAS List of schemas to export (login schema).
STATUS Frequency (secs) job status is to be monitored where
the default (0) will show new status when available.
TABLES Identifies a list of tables to export - one schema only.
TABLESPACES Identifies a list of tablespaces to export.
TRANSPORT_FULL_CHECK Verify storage segments of all tables (N).
TRANSPORT_TABLESPACES List of tablespaces from which metadata will be unloaded.
VERSION Version of objects to export where valid keywords are:
(COMPATIBLE), LATEST, or any valid database version.
The following commands are valid while in interactive mode.
Note: abbreviations are allowed
Command Description
------------------------------------------------------------------------------
ADD_FILE Add dumpfile to dumpfile set.
ADD_FILE=dumpfile-name
CONTINUE_CLIENT Return to logging mode. Job will be re-started if idle.
EXIT_CLIENT Quit client session and leave job running.
HELP Summarize interactive commands.
KILL_JOB Detach and delete job.
PARALLEL Change the number of active workers for current job.
PARALLEL=.
START_JOB Start/resume current job.
STATUS Frequency (secs) job status is to be monitored where
the default (0) will show new status when available.
STATUS=[interval]
STOP_JOB Orderly shutdown of job execution and exits the client.
STOP_JOB=IMMEDIATE performs an immediate shutdown of the
Data Pump job.
Oracle 10g Release 2 (10.2) added the following parameters.
Keyword Description (Default)
------------------------------------------------------------------------------
COMPRESSION Reduce size of dumpfile contents where valid
keyword values are: (METADATA_ONLY) and NONE.
ENCRYPTION_PASSWORD Password key for creating encrypted column data.
SAMPLE Percentage of data to be exported;
The following commands are valid while in interactive mode.
Note: abbreviations are allowed
Command Description
------------------------------------------------------------------------------
FILESIZE Default filesize (bytes) for subsequent ADD_FILE commands.
Oracle 11g Release 1 (11.1) added the following parameters.
Keyword Description (Default)
------------------------------------------------------------------------------
DATA_OPTIONS Data layer flags where the only valid value is:
XML_CLOBS-write XML datatype in CLOB format
ENCRYPTION Encrypt part or all of the dump file where valid keyword
values are: ALL, DATA_ONLY, METADATA_ONLY,
ENCRYPTED_COLUMNS_ONLY, or NONE.
ENCRYPTION_ALGORITHM Specify how encryption should be done where valid
keyword values are: (AES128), AES192, and AES256.
ENCRYPTION_MODE Method of generating encryption key where valid keyword
values are: DUAL, PASSWORD, and (TRANSPARENT).
REMAP_DATA Specify a data conversion function,
e.g. REMAP_DATA=EMP.EMPNO:REMAPPKG.EMPNO.
REUSE_DUMPFILES Overwrite destination dump file if it exists (N).
TRANSPORTABLE Specify whether transportable method can be used where
valid keyword values are: ALWAYS, (NEVER).
The following commands are valid while in interactive mode.
Note: abbreviations are allowed
Command Description
------------------------------------------------------------------------------
REUSE_DUMPFILES Overwrite destination dump file if it exists (N).
Oracle 11g Release 1 (11.2) altered the format of the help output as well as adding the following parameters.
CLUSTER Utilize cluster resources and distribute workers across the Oracle RAC. Valid keyword values are: [Y] and N. SERVICE_NAME Name of an active Service and associated resource group to constrain Oracle RAC resources. SOURCE_EDITION Edition to be used for extracting metadata.
impdp
impdp help=y
Import: Release 10.1.0.2.0 - Production on Saturday, 11 September, 2004 17:22
The Data Pump Import utility provides a mechanism for transferring data objects
between Oracle databases. The utility is invoked with the following command:
Example: impdp scott/tiger DIRECTORY=dmpdir DUMPFILE=scott.dmp
You can control how Import runs by entering the 'impdp' command followed
by various parameters. To specify parameters, you use keywords:
Format: impdp KEYWORD=value or KEYWORD=(value1,value2,...,valueN)
Example: impdp scott/tiger DIRECTORY=dmpdir DUMPFILE=scott.dmp
USERID must be the first parameter on the command line.
Keyword Description (Default)
------------------------------------------------------------------------------
ATTACH Attach to existing job, e.g. ATTACH [=job name].
CONTENT Specifies data to load where the valid keywords are:
(ALL), DATA_ONLY, and METADATA_ONLY.
DIRECTORY Directory object to be used for dump, log, and sql files.
DUMPFILE List of dumpfiles to import from (expdat.dmp),
e.g. DUMPFILE=scott1.dmp, scott2.dmp, dmpdir:scott3.dmp.
ESTIMATE Calculate job estimates where the valid keywords are:
(BLOCKS) and STATISTICS.
EXCLUDE Exclude specific object types, e.g. EXCLUDE=TABLE:EMP.
FLASHBACK_SCN SCN used to set session snapshot back to.
FLASHBACK_TIME Time used to get the SCN closest to the specified time.
FULL Import everything from source (Y).
HELP Display help messages (N).
INCLUDE Include specific object types, e.g. INCLUDE=TABLE_DATA.
JOB_NAME Name of import job to create.
LOGFILE Log file name (import.log).
NETWORK_LINK Name of remote database link to the source system.
NOLOGFILE Do not write logfile.
PARALLEL Change the number of active workers for current job.
PARFILE Specify parameter file.
QUERY Predicate clause used to import a subset of a table.
REMAP_DATAFILE Redefine datafile references in all DDL statements.
REMAP_SCHEMA Objects from one schema are loaded into another schema.
REMAP_TABLESPACE Tablespace object are remapped to another tablespace.
REUSE_DATAFILES Tablespace will be initialized if it already exists (N).
SCHEMAS List of schemas to import.
SKIP_UNUSABLE_INDEXES Skip indexes that were set to the Index Unusable state.
SQLFILE Write all the SQL DDL to a specified file.
STATUS Frequency (secs) job status is to be monitored where
the default (0) will show new status when available.
STREAMS_CONFIGURATION Enable the loading of Streams metadata
TABLE_EXISTS_ACTION Action to take if imported object already exists.
Valid keywords: (SKIP), APPEND, REPLACE and TRUNCATE.
TABLES Identifies a list of tables to import.
TABLESPACES Identifies a list of tablespaces to import.
TRANSFORM Metadata transform to apply (Y/N) to specific objects.
Valid transform keywords: SEGMENT_ATTRIBUTES and STORAGE.
ex. TRANSFORM=SEGMENT_ATTRIBUTES:N:TABLE.
TRANSPORT_DATAFILES List of datafiles to be imported by transportable mode.
TRANSPORT_FULL_CHECK Verify storage segments of all tables (N).
TRANSPORT_TABLESPACES List of tablespaces from which metadata will be loaded.
Only valid in NETWORK_LINK mode import operations.
VERSION Version of objects to export where valid keywords are:
(COMPATIBLE), LATEST, or any valid database version.
Only valid for NETWORK_LINK and SQLFILE.
The following commands are valid while in interactive mode.
Note: abbreviations are allowed
Command Description (Default)11g
------------------------------------------------------------------------------
CONTINUE_CLIENT Return to logging mode. Job will be re-started if idle.
EXIT_CLIENT Quit client session and leave job running.
HELP Summarize interactive commands.
KILL_JOB Detach and delete job.
PARALLEL Change the number of active workers for current job.
PARALLEL=.
START_JOB Start/resume current job.
START_JOB=SKIP_CURRENT will start the job after skipping
any action which was in progress when job was stopped.
STATUS Frequency (secs) job status is to be monitored where
the default (0) will show new status when available.
STATUS=[interval]
STOP_JOB Orderly shutdown of job execution and exits the client.
STOP_JOB=IMMEDIATE performs an immediate shutdown of the
Data Pump job.
Oracle 10g Release 2 (10.2) added the following parameter.
Keyword Description (Default)
------------------------------------------------------------------------------
ENCRYPTION_PASSWORD Password key for accessing encrypted column data.
This parameter is not valid for network import jobs.
Oracle 11g Release 1 (11.1) added the following parameters.
Keyword Description (Default)
------------------------------------------------------------------------------
DATA_OPTIONS Data layer flags where the only valid value is:
SKIP_CONSTRAINT_ERRORS-constraint errors are not fatal.
PARTITION_OPTIONS Specify how partitions should be transformed where the
valid keywords are: DEPARTITION, MERGE and (NONE)
REMAP_DATA Specify a data conversion function,
e.g. REMAP_DATA=EMP.EMPNO:REMAPPKG.EMPNO
Oracle 11g Release 1 (11.2) altered the format of the help output as well as adding the following parameters.
CLUSTER Utilize cluster resources and distribute workers across the Oracle RAC. Valid keyword values are: [Y] and N. SERVICE_NAME Name of an active Service and associated resource group to constrain Oracle RAC resources. SOURCE_EDITION Edition to be used for extracting metadata. TARGET_EDITION Edition to be used for loading metadata.
No comments:
Post a Comment