.\" Automatically generated by Pod::Man 2.16 (Pod::Simple 3.05) .\" .\" Standard preamble: .\" ======================================================================== .de Sh \" Subsection heading .br .if t .Sp .ne 5 .PP \fB\\$1\fR .PP .. .de Sp \" Vertical space (when we can't use .PP) .if t .sp .5v .if n .sp .. .de Vb \" Begin verbatim text .ft CW .nf .ne \\$1 .. .de Ve \" End verbatim text .ft R .fi .. .\" Set up some character translations and predefined strings. \*(-- will .\" give an unbreakable dash, \*(PI will give pi, \*(L" will give a left .\" double quote, and \*(R" will give a right double quote. \*(C+ will .\" give a nicer C++. Capital omega is used to do unbreakable dashes and .\" therefore won't be available. \*(C` and \*(C' expand to `' in nroff, .\" nothing in troff, for use with C<>. .tr \(*W- .ds C+ C\v'-.1v'\h'-1p'\s-2+\h'-1p'+\s0\v'.1v'\h'-1p' .ie n \{\ . ds -- \(*W- . ds PI pi . if (\n(.H=4u)&(1m=24u) .ds -- \(*W\h'-12u'\(*W\h'-12u'-\" diablo 10 pitch . if (\n(.H=4u)&(1m=20u) .ds -- \(*W\h'-12u'\(*W\h'-8u'-\" diablo 12 pitch . ds L" "" . ds R" "" . ds C` "" . ds C' "" 'br\} .el\{\ . ds -- \|\(em\| . ds PI \(*p . ds L" `` . ds R" '' 'br\} .\" .\" Escape single quotes in literal strings from groff's Unicode transform. .ie \n(.g .ds Aq \(aq .el .ds Aq ' .\" .\" If the F register is turned on, we'll generate index entries on stderr for .\" titles (.TH), headers (.SH), subsections (.Sh), items (.Ip), and index .\" entries marked with X<> in POD. Of course, you'll have to process the .\" output yourself in some meaningful fashion. .ie \nF \{\ . de IX . tm Index:\\$1\t\\n%\t"\\$2" .. . nr % 0 . rr F .\} .el \{\ . de IX .. .\} .\" .\" Accent mark definitions (@(#)ms.acc 1.5 88/02/08 SMI; from UCB 4.2). .\" Fear. Run. Save yourself. No user-serviceable parts. . \" fudge factors for nroff and troff .if n \{\ . ds #H 0 . ds #V .8m . ds #F .3m . ds #[ \f1 . ds #] \fP .\} .if t \{\ . ds #H ((1u-(\\\\n(.fu%2u))*.13m) . ds #V .6m . ds #F 0 . ds #[ \& . ds #] \& .\} . \" simple accents for nroff and troff .if n \{\ . ds ' \& . ds ` \& . ds ^ \& . ds , \& . ds ~ ~ . ds / .\} .if t \{\ . ds ' \\k:\h'-(\\n(.wu*8/10-\*(#H)'\'\h"|\\n:u" . ds ` \\k:\h'-(\\n(.wu*8/10-\*(#H)'\`\h'|\\n:u' . ds ^ \\k:\h'-(\\n(.wu*10/11-\*(#H)'^\h'|\\n:u' . ds , \\k:\h'-(\\n(.wu*8/10)',\h'|\\n:u' . ds ~ \\k:\h'-(\\n(.wu-\*(#H-.1m)'~\h'|\\n:u' . ds / \\k:\h'-(\\n(.wu*8/10-\*(#H)'\z\(sl\h'|\\n:u' .\} . \" troff and (daisy-wheel) nroff accents .ds : \\k:\h'-(\\n(.wu*8/10-\*(#H+.1m+\*(#F)'\v'-\*(#V'\z.\h'.2m+\*(#F'.\h'|\\n:u'\v'\*(#V' .ds 8 \h'\*(#H'\(*b\h'-\*(#H' .ds o \\k:\h'-(\\n(.wu+\w'\(de'u-\*(#H)/2u'\v'-.3n'\*(#[\z\(de\v'.3n'\h'|\\n:u'\*(#] .ds d- \h'\*(#H'\(pd\h'-\w'~'u'\v'-.25m'\f2\(hy\fP\v'.25m'\h'-\*(#H' .ds D- D\\k:\h'-\w'D'u'\v'-.11m'\z\(hy\v'.11m'\h'|\\n:u' .ds th \*(#[\v'.3m'\s+1I\s-1\v'-.3m'\h'-(\w'I'u*2/3)'\s-1o\s+1\*(#] .ds Th \*(#[\s+2I\s-2\h'-\w'I'u*3/5'\v'-.3m'o\v'.3m'\*(#] .ds ae a\h'-(\w'a'u*4/10)'e .ds Ae A\h'-(\w'A'u*4/10)'E . \" corrections for vroff .if v .ds ~ \\k:\h'-(\\n(.wu*9/10-\*(#H)'\s-2\u~\d\s+2\h'|\\n:u' .if v .ds ^ \\k:\h'-(\\n(.wu*10/11-\*(#H)'\v'-.4m'^\v'.4m'\h'|\\n:u' . \" for low resolution devices (crt and lpr) .if \n(.H>23 .if \n(.V>19 \ \{\ . ds : e . ds 8 ss . ds o a . ds d- d\h'-1'\(ga . ds D- D\h'-1'\(hy . ds th \o'bp' . ds Th \o'LP' . ds ae ae . ds Ae AE .\} .rm #[ #] #H #V #F C .\" ======================================================================== .\" .IX Title "Roadmap 3" .TH Roadmap 3 "2008-03-01" "perl v5.10.0" "User Contributed Perl Documentation" .\" For nroff, turn off justification. Always turn off hyphenation; it makes .\" way too many mistakes in technical documents. .if n .ad l .nh .SH "NAME" DBI::Roadmap \- Planned Enhancements for the DBI .PP Tim Bunce \- 12th November 2004 .SH "SYNOPSIS" .IX Header "SYNOPSIS" This document gives a high level overview of the future of the Perl \&\s-1DBI\s0 module. .PP The \s-1DBI\s0 module is the standard database interface for Perl applications. It is used worldwide in countless applications, in every kind of business, and on platforms from clustered super-computers to PDAs. Database interface drivers are available for all common databases and many not-so-common ones. .PP The planned enhancements cover testing, performance, high availability and load balancing, batch statements, Unicode, database portability, and more. .PP Addressing these issues together, in coordinated way, will help ensure maximum future functionality with minimal disruptive (incompatible) upgrades. .SH "SCOPE" .IX Header "SCOPE" Broad categories of changes are outlined here along with some rationale, but implementation details and minor planned enhancements are omitted. More details can be found in: .SH "CHANGES AND ENHANCEMENTS" .IX Header "CHANGES AND ENHANCEMENTS" These are grouped into categories and are not listed in any particular order. .Sh "Performance" .IX Subsection "Performance" The \s-1DBI\s0 has always treated performance as a priority. Some parts of the implementation, however, remain unoptimized, especially in relation to threads. .PP * When the \s-1DBI\s0 is used with a Perl built with thread support enabled (such as for Apache mod_perl 2, and some common Linux distributions) it runs significantly slower. There are two reasons for this and both can be fixed but require non-trivial changes to both the \s-1DBI\s0 and drivers. .PP * Connection pooling in a threaded application, such as mod_perl, is difficult because \s-1DBI\s0 handles cannot be passed between threads. An alternative mechanism for passing connections between threads has been defined, and an experimental connection pool module implemented using it, but development has stalled. .PP * The majority of \s-1DBI\s0 handle creation code is implemented in Perl. Moving most of this to C will speed up handle creation significantly. .PP * The popular \fIfetchrow_hashref()\fR method is many times slower than \&\fIfetchrow_arrayref()\fR. It has to get the names of the columns, then create and load a new hash each time. A \f(CW$h\fR\->{FetchHashReuse} attribute would allow the same hash to be reused each time making \fIfetchrow_hashref()\fR about the same speed as \fIfetchrow_arrayref()\fR. .PP * Support for asynchronous (non-blocking) \s-1DBI\s0 method calls would enable applications to continue processing in parallel with database activity. This is also relevant for \s-1GUI\s0 and other event-driven applications. The \s-1DBI\s0 needs to define a standard interface for this so drivers can implement it in a portable way, where possible. .PP These changes would significantly enhance the performance of the \&\s-1DBI\s0 and many applications which use the \s-1DBI\s0. .Sh "Testing" .IX Subsection "Testing" The \s-1DBI\s0 has a test suite. Every driver has a test suite. Each is limited in its scope. The driver test suite is testing for behavior that the driver author \fIthinks\fR the \s-1DBI\s0 specifies, but may be subtly incorrect. These test suites are poorly maintained because the return on investment for a single driver is too low to provide sufficient incentive. .PP A common test suite that can be reused by all the drivers is needed. It would: .PP * Improve the quality of the \s-1DBI\s0 and drivers. .PP * Ensure all drivers conform to the \s-1DBI\s0 specification. Easing the porting of applications between databases, and the implementation of database independent modules layered over the \s-1DBI\s0. .PP * Improve the \s-1DBI\s0 specification by clarifying unclear issues in order to implement test cases. .PP * Encourage expansion of the test suite as driver authors and others will be motivated by the greater benefits of their contributions. .PP * Detect and record optional functionality that a driver has not yet implemented. .PP * Improve the testing of \s-1DBI\s0 subclassing, DBI::PurePerl and the various \*(L"transparent\*(R" drivers, such as DBD::Proxy and DBD::Multiplex, by automatically running the test suite through them. .PP These changes would improve the quality of all applications using the \s-1DBI\s0. .Sh "High Availability and Load Balancing" .IX Subsection "High Availability and Load Balancing" * The DBD::Multiplex driver provides a framework to enable a wide range of dynamic functionality, including support for high-availability, failover, load-balancing, caching, and access to distributed data. It is currently being enhanced but development has stalled. .PP * The DBD::Proxy module is complex and relatively inefficient because it's trying to be a complete proxy for most \s-1DBI\s0 method calls. For many applications a simpler proxy architecture that operates with a single round-trip to the server would be simpler, faster, and more flexible. .PP New proxy client and server classes are needed, which could be subclassed to support specific client to server transport mechanisms (such as \s-1HTTP\s0 and Spread::Queue). Apart from the efficiency gains, this would also enable the use of a load-balanced pool of stateless servers for greater scalability and reliability. .PP * The \s-1DBI\s0 currently offers no support for distributed transactions. The most useful elements of the standard \s-1XA\s0 distributed transaction interface standard could be included in the \s-1DBI\s0 specification. Drivers for databases which support distributed transactions could then be extended to support it. .PP These changes would enable new kinds of \s-1DBI\s0 applications for critical environments. .Sh "Unicode" .IX Subsection "Unicode" Use of Unicode with the \s-1DBI\s0 is growing rapidly. The \s-1DBI\s0 should do more to help drivers support Unicode and help applications work with drivers that don't yet support Unicode directly. .PP * Define expected behavior for fetching data and binding parameters. .PP * Provide interfaces to support Unicode issues for \s-1XS\s0 and pure Perl drivers and applications. .PP * Provide functions for applications to help diagnose inconsistencies between byte string contents and setting of the SvUTF8 flag. .PP These changes would smooth the transition to Unicode for many applications and drivers. .Sh "Batch Statements" .IX Subsection "Batch Statements" Batch statements are a sequence of \s-1SQL\s0 statements, or a stored procedure containing a sequence of \s-1SQL\s0 statements, which can be executed as a whole. .PP Currently the \s-1DBI\s0 has no standard interface for dealing with multiple results from batch statements. After considerable discussion, an interface design has been agreed upon with driver authors, but has not yet been implemented. .PP These changes would enable greater application portability between databases, and greater performance for databases that directly support batch statements. .Sh "Introspection" .IX Subsection "Introspection" * The methods of the \s-1DBI\s0 \s-1API\s0 are installed dynamically when the \s-1DBI\s0 is loaded. The data structure used to define the methods and their dispatch behavior should be made part of the \s-1DBI\s0 \s-1API\s0. This would enable more flexible and correct behavior by modules subclassing the \s-1DBI\s0 and by dynamic drivers such as DBD::Proxy and DBD::Multiplex. .PP * Handle attribute information should also be made available, for the same reasons. .PP * Currently is it not possible to discover all the child statement handles that belong to a database handle (or all database handles that belong to a driver handle). This makes certain tasks more difficult, especially some debugging scenarios. A cache of weak references to child handles would solve the problem without creating reference loops. .PP * It is often useful to know which handle attributes have been changed since the handle was created (e.g., in mod_perl where a handle needs to be reset or cloned). This will become more important as developers start exploring use of the newly added \&\f(CW$h1\fR\->swap_inner_handle($h2) method. .PP These changes would simplify and improve the stability of many advanced uses of the \s-1DBI\s0. .Sh "Extensibility" .IX Subsection "Extensibility" The \s-1DBI\s0 can be extended in three main dimensions: subclassing the \&\s-1DBI\s0, subclassing a driver, and callback hooks. Each has different pros and cons, each is applicable in different situations, and all need enhancing. .PP * Subclassing the \s-1DBI\s0 is functional but not well defined and some key elements are incomplete, particularly the DbTypeSubclass mechanism (that automatically subclasses to a class tree according to the type of database being used). It also needs more thorough testing. .PP * Subclassing a driver is undocumented, poorly tested and very probably incomplete. However it's a powerful way to embed certain kinds of functionality 'below' applications while avoiding some of the side-effects of subclassing the \s-1DBI\s0 (especially in relation to error handling). .PP * Callbacks are currently limited to error handling (the HandleError and HandleSetError attributes). Providing callback hooks for more events, such as a row being fetched, would enable utility modules, for example, to modify the behavior of a handle independent of any subclassing in use. .PP These changes would enable cleaner and more powerful integration between applications, layered modules, and the \s-1DBI\s0. .Sh "Debugability" .IX Subsection "Debugability" * Enabling \s-1DBI\s0 trace output at a high level of detail causes a large volume of output, much of it probably unrelated to the problem being investigated. Trace output should be controlled by the new named-topic mechanism instead of just the trace level. .PP * Calls to \s-1XS\s0 functions (such as many \s-1DBI\s0 and driver methods) don't normally appear in the call stack. Optionally enabling that would enable more useful diagnostics to be produced. .PP * Integration with the Perl debugger would make it simpler to perform actions on a per-handle basis (such as breakpoint on execute, breakpoint on error). .PP These changes would enable more rapid application development and fault finding. .Sh "Database Portability" .IX Subsection "Database Portability" * The \s-1DBI\s0 has not yet addressed the issue of portability among \s-1SQL\s0 dialects. This is the main hurdle limiting database portability for \s-1DBI\s0 applications. .PP The goal is \fInot\fR to fully parse the \s-1SQL\s0 and rewrite it in a different dialect. That's well beyond the scope of the \s-1DBI\s0 and should be left to layered modules. A simple token rewriting mechanism for five comment styles, two quoting styles, four placeholder styles, plus the \s-1ODBC\s0 \*(L"{foo ...}\*(R" escape syntax, is sufficient to significantly raise the level of \s-1SQL\s0 portability. .PP * Another problem area is date/time formatting. Since version 1.41 the \s-1DBI\s0 has defined a way to express that dates should be fetched in \s-1SQL\s0 standard date format (YYYY-MM-DD). This is one example of the more general case where \fIbind_col()\fR needs to be called with particular attributes on all columns of a particular type. .PP A mechanism is needed whereby an application can specify default \&\fIbind_col()\fR attributes to be applied automatically for each column type. With a single step, all \s-1DATE\s0 type columns, for example, can be set to be returned in the standard format. .PP These changes would enable greater database portability for applications and greater functionality for layered modules. .Sh "Intellectual Property" .IX Subsection "Intellectual Property" * Clarify current intellectual property status, including a review of past contributions to ensure the \s-1DBI\s0 is unemcumbered. .PP * Establish a procedure for vetting future contributions for any intellectual property issues. .PP These changes are important for companies taking a formal approach to assessing their risks in using Open Source software. .Sh "Other Enhancements" .IX Subsection "Other Enhancements" * Reduce the work needed to create new database interface drivers. .PP * Definition of an interface to support scrollable cursors. .Sh "Parrot and Perl 6" .IX Subsection "Parrot and Perl 6" The current \s-1DBI\s0 implementation in C code is unlikely to run on Perl 6. Perl 6 will target the Parrot virtual machine and so the internal architecture will be radically different from Perl 5. .PP One of the goals of the Parrot project is to be a platform for many dynamic languages (including Python, \s-1PHP\s0, Ruby, etc) and to enable those languages to reuse each others modules. A database interface for Parrot is also a database interface for any and all languages that run on Parrot. .PP The Perl \s-1DBI\s0 would make an excellent base for a Parrot database interface because it has more functionality, and is more mature and extensible, than the database interfaces of the other dynamic languages. .PP I plan to better define the \s-1API\s0 between the \s-1DBI\s0 and the drivers and use that \s-1API\s0 as the primary \s-1API\s0 for the 'raw' Parrot database interface. This project is known a Parrot \s-1DBDI\s0 (for \*(L"DataBase Driver Interface\*(R"). The announcement can be read in .PP The bulk of the work will be translating the \s-1DBI\s0 C and Perl base class code into Parrot \s-1PIR\s0, or a suitable language that generates \&\s-1PIR\s0. The project stalled, due to Parrot not having key functionality at the time. That has been resolved but the project has not yet restarted. .PP Each language targeting Parrot would implement their own small \&'thin' language-specific method dispatcher (a \*(L"Perl6 \s-1DBI\s0\*(R", \*(L"Python \&\s-1DBI\s0\*(R", \*(L"\s-1PHP\s0 \s-1DBI\s0\*(R" etc) layered over the common Parrot \s-1DBDI\s0 interface and drivers. .PP The major benefit of the \s-1DBDI\s0 project is that a much wider community of developers share the same database drivers. There would be more developers maintaining less code so the benefits of the Open Source model are magnified. .SH "PRIORITIES" .IX Header "PRIORITIES" .Sh "Transition Drivers" .IX Subsection "Transition Drivers" The first priority is to make all the infrastructure changes that impact drivers and make an alpha release available for driver authors. .PP As far as possible, the changes will be implemented in a way that enables driver authors use the same code base for \s-1DBI\s0 v1 and \s-1DBI\s0 v2. .PP The main changes required by driver authors are: .PP * Code changes for \s-1PERL_NO_GET_CONTEXT\s0, plus removing \s-1PERL_POLLUTE\s0 and \s-1DBIS\s0 .PP * Code changes in \s-1DBI/DBD\s0 interface (new way to create handles, new callbacks etc) .PP * Common test suite infrastructure (driver-specific test base class) .Sh "Transition Applications" .IX Subsection "Transition Applications" A small set of incompatible changes that may impact some applications will also be made in v2.0. See http://svn.perl.org/modules/dbi/trunk/ToDo .Sh "Incremental Developments" .IX Subsection "Incremental Developments" Once \s-1DBI\s0 v2.0 is available, the other enhancements can be implemented incrementally on the updated foundations. Priorities for those changes have not been set. .Sh "\s-1DBI\s0 v1" .IX Subsection "DBI v1" \&\s-1DBI\s0 v1 will continue to be maintained on a separate branch for bug fixes and any enhancements that ease the transition to \s-1DBI\s0 v2. .SH "RESOURCES AND CONTRIBUTIONS" .IX Header "RESOURCES AND CONTRIBUTIONS" See for \fIhow you can help\fR. .PP If your company has benefited from the \s-1DBI\s0, please consider if it could make a donation to The Perl Foundation \*(L"\s-1DBI\s0 Development\*(R" fund at to secure future development. .PP Alternatively, if your company would benefit from a specific new \&\s-1DBI\s0 feature, please consider sponsoring its development through my consulting company, Data Plan Services. Work is performed rapidly on a fixed-price payment-on-delivery basis. Contact me for details. .PP Using such targeted financing allows you to contribute to \s-1DBI\s0 development and rapidly get something specific and directly valuable to you in return. .PP My company also offers annual support contracts for the \s-1DBI\s0, which provide another way to support the \s-1DBI\s0 and get something specific in return. Contact me for details. .PP Thank you.