-
-
Notifications
You must be signed in to change notification settings - Fork 0
/
news.xml
1963 lines (1941 loc) · 236 KB
/
news.xml
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
<?xml version="1.0" encoding="utf-8"?>
<feed xmlns="http://www.w3.org/2005/Atom"><title>News</title><link href="https://www.psycopg.org/articles/" rel="alternate"></link><link href="https://www.psycopg.org/news.xml" rel="self"></link><id>urn:uuid:1f583092-c4bb-3a77-9f6f-c8ac41335aab</id><updated>2024-06-30T00:00:00Z</updated><author><name></name></author><entry><title>Psycopg 3.2 released</title><link href="https://www.psycopg.org/articles/2024/06/30/psycopg-32-released/" rel="alternate"></link><updated>2024-06-30T00:00:00Z</updated><author><name>Daniele Varrazzo</name></author><id>urn:uuid:31cb8a57-f930-3fb0-82c6-4067eb76bd76</id><content type="html"><p>It was quite the ride! But we made it!</p>
<p>After almost two years, 846 commits, more than 700 new tests, more than 20000
changes in 310 files (I didn't even realise that there were 310 files in this
project...) Psycopg 3.2 has been released!</p>
<p>This release brings a few new feature and hopefully no meaningful non-backward
compatible change. The whole list of changes is available <a class="reference external" href="https://www.psycopg.org/psycopg3/docs/news.html#psycopg-3-2">in the changelog</a>; these are
some of the major points explained.</p>
<!-- CUT-HERE -->
<div class="section" id="numpy-scalars-support">
<h2>Numpy scalars support</h2>
<p>In many scientific applications, <a class="reference external" href="https://numpy.org/doc/stable/reference/arrays.scalars.html#built-in-scalar-types">Numpy scalars</a>
are widely used, either by themselves or in conjunction with regular Python
values. However there was no support for storing them to the database and a
conversion to normal Python values was necessary. Starting from Psycopg 3.2
storing Numpy scalars is automatic and the operation efficient.</p>
<p>A natural extension would be to convert between Numpy and PostgreSQL arrays
too. However there hasn't been much demand for the feature, therefore it's
currently <a class="reference external" href="https://github.com/psycopg/psycopg/issues/336">on the back burner</a> but can be implemented if
there is demand.</p>
</div>
<div class="section" id="postgresql-parameters">
<h2>PostgreSQL parameters</h2>
<p>Psycopg uses placeholders such as <tt class="docutils literal">%s</tt> and <tt class="docutils literal">%(name)s</tt> to <a class="reference external" href="https://www.psycopg.org/psycopg3/docs/basic/params.html">pass values to
queries</a>. These
formats are familiar to Python developers, but they are quite foreign in
PostgreSQL environment, because, natively, <a class="reference external" href="https://www.postgresql.org/docs/current/libpq-exec.html#LIBPQ-PQEXECPARAMS">PostgreSQL uses a number-based
placeholder format</a>
(such as <tt class="docutils literal">$1</tt>, <tt class="docutils literal">$2</tt>...) Psycopg, internally, converts the first format
into the second.</p>
<p>It is now possible to execute queries using the PostgreSQL format by using the
<a class="reference external" href="https://www.psycopg.org/psycopg3/docs/advanced/cursors.html#raw-query-cursors">raw query cursors</a>,
which should feel more familiar to PostgreSQL developers and maybe lower the
barrier to convert programs using large bodies of native queries to Python
(the PostgreSQL test suite, maybe?)</p>
<pre class="code python literal-block">
<span class="n">cur</span> <span class="o">=</span> <span class="n">psycopg</span><span class="o">.</span><span class="n">RawCursor</span><span class="p">(</span><span class="n">conn</span><span class="p">)</span><span class="w">
</span><span class="n">cur</span><span class="o">.</span><span class="n">execute</span><span class="p">(</span><span class="s2">&quot;SELECT ($1 + $2) * $1&quot;</span><span class="p">,</span> <span class="p">[</span><span class="mi">3</span><span class="p">,</span> <span class="mi">5</span><span class="p">])</span><span class="o">.</span><span class="n">fetchone</span><span class="p">()</span><span class="w">
</span><span class="p">(</span><span class="mi">24</span><span class="p">,)</span>
</pre>
</div>
<div class="section" id="scalar-row-factory">
<h2>Scalar row factory</h2>
<p>The example above shows a pretty common annoyance. How many times do you need
a single value from the database and you are returned a tuple?</p>
<p>Psycopg normally emits records as Python tuples; the behaviour can be
customized to return named tuples, dictionaries, or entirely custom objects
with the use of <a class="reference external" href="https://www.psycopg.org/psycopg3/docs/advanced/rows.html">row factories</a>.</p>
<p>In the frequent case of a query returning a single value, the new <a class="reference external" href="https://www.psycopg.org/psycopg3/docs/api/rows.html#psycopg.rows.scalar_row">scalar_row</a>
factory will return only that:</p>
<pre class="code python literal-block">
<span class="n">cur</span> <span class="o">=</span> <span class="n">psycopg</span><span class="o">.</span><span class="n">RawCursor</span><span class="p">(</span><span class="n">conn</span><span class="p">,</span> <span class="n">row_factory</span><span class="o">=</span><span class="n">scalar_row</span><span class="p">)</span><span class="w">
</span><span class="n">cur</span><span class="o">.</span><span class="n">execute</span><span class="p">(</span><span class="s2">&quot;SELECT ($1 + $2) * $1&quot;</span><span class="p">,</span> <span class="p">[</span><span class="mi">3</span><span class="p">,</span> <span class="mi">5</span><span class="p">])</span><span class="o">.</span><span class="n">fetchone</span><span class="p">()</span><span class="w">
</span><span class="mi">24</span>
</pre>
<p>This is not a feature of <tt class="docutils literal">RawCursor</tt> only, but it's independent from the
choice of the cursor class. We just needed to fix the example above!</p>
</div>
<div class="section" id="libpq-17-features">
<h2>Libpq 17 features</h2>
<p>In the upcoming PostgreSQL 17 release, the libpq (the PostgreSQL client
library used internally by Psycopg) has seen an unusually intense activity,
with the introduction of <a class="reference external" href="https://www.postgresql.org/docs/17/release-17.html#RELEASE-17-LIBPQ">several new features</a>.</p>
<p>Our friend Denis Laxalde has been quick to build features and improvements on
top of these new functionalities. So, when Psycopg is used with libpq 17, it
can benefit of features such as:</p>
<ul class="simple">
<li><a class="reference external" href="https://www.psycopg.org/psycopg3/docs/api/connections.html#psycopg.Connection.cancel_safe">asynchronous, safe cancellation</a></li>
<li><a class="reference external" href="https://www.psycopg.org/psycopg3/docs/api/cursors.html#psycopg.Cursor.stream">chunked stream results</a></li>
<li><a class="reference external" href="https://www.psycopg.org/psycopg3/docs/advanced/prepare.html#pgbouncer">better interaction with PgBouncer</a></li>
</ul>
<p>A new <a class="reference external" href="https://www.psycopg.org/psycopg3/docs/api/objects.html#psycopg.Capabilities">capabilities object</a>
can help to navigate the differences and to write programs either degrading
gracefully or crashing helpfully if the libpq used doesn't offer a requested
functionality.</p>
</div>
<div class="section" id="easier-interaction-with-notifications">
<h2>Easier interaction with notifications</h2>
<p>Psycopg 3 introduced a <a class="reference external" href="https://www.psycopg.org/psycopg3/docs/advanced/async.html#asynchronous-notifications">notifications generator</a>
to receive asynchronous notification from the database. However the generator
turned out to be... difficult to stop! It could be stopped upon receiving a
specific notification as a message, but, because of Python quirks, not easily
from the rest of the program.</p>
<pre class="code python literal-block">
<span class="kn">import</span> <span class="nn">psycopg</span><span class="w">
</span><span class="n">conn</span> <span class="o">=</span> <span class="n">psycopg</span><span class="o">.</span><span class="n">connect</span><span class="p">(</span><span class="s2">&quot;&quot;</span><span class="p">,</span> <span class="n">autocommit</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span><span class="w">
</span><span class="n">conn</span><span class="o">.</span><span class="n">execute</span><span class="p">(</span><span class="s2">&quot;LISTEN mychan&quot;</span><span class="p">)</span><span class="w">
</span><span class="n">gen</span> <span class="o">=</span> <span class="n">conn</span><span class="o">.</span><span class="n">notifies</span><span class="p">()</span><span class="w">
</span><span class="k">for</span> <span class="n">notify</span> <span class="ow">in</span> <span class="n">gen</span><span class="p">:</span><span class="w">
</span> <span class="nb">print</span><span class="p">(</span><span class="n">notify</span><span class="p">)</span><span class="w">
</span> <span class="c1"># ehm... please kill me!</span>
</pre>
<p>New <tt class="docutils literal">timeout</tt> and <tt class="docutils literal">stop_after</tt> parameters allow for better control of a
notification listening task (often a component of larger applications) and to
provide better ways to control its operations. Such as to kindly tell it that
its services are not requested anymore without having to kill the whole
program!</p>
</div>
<div class="section" id="less-work-for-us">
<h2>Less work for us!</h2>
<p>An interesting internal change has helped us to reduce the amount of code to
write and maintain.</p>
<p>All the Psycopg objects interacting with the network come in two flavours: one
implementing &quot;classic&quot; blocking methods (with which concurrency in a process
can be implemented via multi-threading) and one implementing <a class="reference external" href="https://www.psycopg.org/psycopg3/docs/advanced/async.html#asynchronous-operations">asynchronous
methods</a>
to participate in <a class="reference external" href="https://docs.python.org/3/library/asyncio.html">collaborative concurrency</a>.</p>
<p>Thanks to an early design choice, all the libpq I/O interaction only happens
via asynchronous functions and is shared by both the sync and the async
objects; however the code implementing the outermost objects and highest level
behaviour had to be pretty much almost duplicated, with the same features
implemented almost identically with and without async/await keywords, bugs to
be tested and fixed on two sides...</p>
<p>We have therefore developed an <a class="reference external" href="https://github.com/psycopg/psycopg/blob/3.2.0/tools/async_to_sync.py">async_to_sync conversion tool</a> to
generate the synchronous code starting from the AST of the asynchronous
counterpart. As a result, the 20-25% of the codebase is now automatically
generated and doesn't require specific maintenance. The process of converting
the sync side from hand-written to auto-generated has also highlighted subtle
differences between async and sync behaviours, which have been addressed, and
affects tests too.</p>
<p>The technique could be useful for other projects maintaining both sync and
async code, and is interesting enough to require an article of its own to
be written...</p>
</div>
<div class="section" id="we-need-your-help">
<h2>We need your help!</h2>
<p>Psycopg, first v2, now v3, is the de-facto standard for the communication
between Python and PostgreSQL, two major components of innumerable businesses
and mission-critical infrastructures.</p>
<p>Maintaining such a critical library to the highest standard of reliability,
completeness, performance requires a lot of care and work.</p>
<p>If you are a Python and PostgreSQL user and would like to make sure that the
interface between the two is well maintained and continuously improved, please
consider <a class="reference external" href="https://github.com/sponsors/dvarrazzo">sponsoring the project</a>
and to be one of <a class="reference external" href="https://www.psycopg.org/sponsors/">our sponsors</a> 💜</p>
<p>Thank you very much, happy hacking!</p>
</div>
</content></entry><entry><title>Psycopg 3.1 released</title><link href="https://www.psycopg.org/articles/2022/08/30/psycopg-31-released/" rel="alternate"></link><updated>2022-08-30T00:00:00Z</updated><author><name>Daniele Varrazzo</name></author><id>urn:uuid:bca301bf-8fa7-306d-a4d6-022a4af604f5</id><content type="html"><p>Hello,</p>
<p>After several months of development, we are proud to release Psycopg 3.1!</p>
<p>Psycopg 3.1 is a gradual improvement on Psycopg 3.0, introducing new exciting
features, redefining what can be done on the boundary between Python and
PostgreSQL.</p>
<!-- CUT-HERE -->
<div class="section" id="pipeline-mode">
<h2>Pipeline mode</h2>
<p>The <a class="reference external" href="https://www.psycopg.org/psycopg3/docs/advanced/pipeline.html">pipeline mode</a>
is by far the biggest feature introduced in Psycopg 3.1, largely the work of
Denis Laxalde and supported by <a class="reference external" href="http://www.dalibo.com">Dalibo</a>. In
pipeline mode, Psycopg will send batches of commands to the server without
waiting for a response for every operation, resulting in a massive speed
improvement.</p>
<p>The pipeline mode is exposed to Python as a context block. Within the block,
Psycopg will manage the pipeline in a transparent way, even allowing the use of
features which break the pipeline flow, for instance to fetch results, or to
manage transactions:</p>
<pre class="code python literal-block">
<span class="k">with</span> <span class="n">conn</span><span class="o">.</span><span class="n">pipeline</span><span class="p">():</span><span class="w">
</span> <span class="n">cur</span> <span class="o">=</span> <span class="n">conn</span><span class="o">.</span><span class="n">cursor</span><span class="p">()</span><span class="w">
</span> <span class="k">for</span> <span class="n">op</span> <span class="ow">in</span> <span class="n">operations</span><span class="p">:</span><span class="w">
</span> <span class="n">cur</span><span class="o">.</span><span class="n">execute</span><span class="p">(</span><span class="n">op</span><span class="p">)</span><span class="w">
</span> <span class="k">with</span> <span class="n">conn</span><span class="o">.</span><span class="n">transaction</span><span class="p">():</span><span class="w">
</span> <span class="n">cur</span><span class="o">.</span><span class="n">execute</span><span class="p">(</span><span class="n">op1</span><span class="p">)</span><span class="w">
</span> <span class="n">rec</span> <span class="o">=</span> <span class="n">cur</span><span class="o">.</span><span class="n">fetchone</span><span class="p">()</span><span class="w">
</span> <span class="n">cur</span><span class="o">.</span><span class="n">execute</span><span class="p">(</span><span class="n">op2</span><span class="p">,</span> <span class="p">[</span><span class="n">rec</span><span class="o">.</span><span class="n">id</span><span class="p">])</span><span class="w">
</span> <span class="o">...</span>
</pre>
<p>Quantifying the speedup is difficult, as it depends on the network conditions
and on the pattern of statements executed. In particularly bad conditions (250
ms of ping time between client and server), we measured a loop of 100 inserts
to take 25 s in normal mode and just 0.5 s in pipeline mode, for a 50x
speedup. Testing the same operation on a localhost connection, we have
measured a &gt;20x speedup (a 5000 inserts batch taking 0.3 s instead of 6.6 s).
If the program requires results from the server before sending further
statement (for instance to insert in a table and then use the new record's
primary key to insert related records in different tables) you can expect
less dramatic speedups.</p>
</div>
<div class="section" id="client-binding-cursors">
<h2>Client-binding cursors</h2>
<p>Psycopg 3 uses <a class="reference external" href="https://www.psycopg.org/psycopg3/docs/basic/from_pg2.html#server-side-binding">server-side binding</a>,
passing the query and adapted arguments separately. This allows to use several
features otherwise unavailable, such as prepared statements. However, many
types of statements, especially data-definition, don't support server-side
parameters. The <a class="reference external" href="https://www.psycopg.org/psycopg3/docs/api/sql.html">'sql' module</a> allows to compose
statements on the client, but using it might require many changes to programs
making heavy use of data-definition statements.</p>
<p>Psycopg 3.1 introduces a <a class="reference external" href="https://www.psycopg.org/psycopg3/docs/advanced/cursors.html#client-side-binding-cursors">ClientCursor</a>
object, which makes psycopg2 programs easier to port. This cursor reproduces
psycopg2's way of composing queries, which is not the most efficient, but for
certain programs it is exactly what is needed.</p>
<pre class="code python literal-block">
<span class="k">with</span> <span class="n">psycopg</span><span class="o">.</span><span class="n">connect</span><span class="p">(</span><span class="o">...</span><span class="p">,</span> <span class="n">cursor_factory</span><span class="o">=</span><span class="n">ClientCursor</span><span class="p">)</span> <span class="k">as</span> <span class="n">conn</span><span class="p">:</span><span class="w">
</span> <span class="n">cur</span> <span class="o">=</span> <span class="n">conn</span><span class="o">.</span><span class="n">cursor</span><span class="p">()</span> <span class="c1"># this is a client-side binding cursor</span><span class="w">
</span> <span class="c1"># This statement doesn't support server-side parameters</span><span class="w">
</span> <span class="n">cur</span><span class="o">.</span><span class="n">execute</span><span class="p">(</span><span class="s2">&quot;ALTER TABLE x ALTER y SET DEFAULT </span><span class="si">%s</span><span class="s2">&quot;</span><span class="p">,</span> <span class="p">[</span><span class="s2">&quot;hello&quot;</span><span class="p">])</span>
</pre>
<p>As a bonus, the client cursor reintroduces the handy and often requested
<a class="reference external" href="https://www.psycopg.org/psycopg3/docs/api/cursors.html#psycopg.ClientCursor.mogrify">mogrify()</a>
method, which returns the query merged with the parameters the way it is
passed to the server:</p>
<pre class="code python literal-block">
<span class="n">cur</span><span class="o">.</span><span class="n">mogrify</span><span class="p">(</span><span class="s2">&quot;ALTER TABLE x ALTER y SET DEFAULT </span><span class="si">%s</span><span class="s2">&quot;</span><span class="p">,</span> <span class="p">[</span><span class="s2">&quot;hell'o&quot;</span><span class="p">])</span><span class="w">
</span><span class="s2">&quot;ALTER TABLE x ALTER y SET DEFAULT 'hell''o'&quot;</span>
</pre>
</div>
<div class="section" id="enum-adaptation">
<h2>Enum adaptation</h2>
<p>Python has <a class="reference external" href="https://docs.python.org/3/library/enum.html">enums</a>, PostgreSQL
has <a class="reference external" href="https://www.postgresql.org/docs/current/static/datatype-enum.html">enums</a>...
Why not map them into each other? Well for a start because they are
actually pretty different from each other (Python enums have a type and value,
Postgres ones are just identities) and because often, in programs,
differences between the enums in the db and the code creep in.</p>
<p>Psycopg 3.1 introduces <a class="reference external" href="https://www.psycopg.org/psycopg3/docs/basic/adapt.html#enum-adaptation">a flexible adapter between Python and Postgres enums</a>.
It can be used in a simple way when there is a one-to-one mapping between the
enums:</p>
<pre class="code literal-block">
=# CREATE TYPE numbers AS ENUM ('ONE', 'TWO', 'THREE');
</pre>
<pre class="code python literal-block">
<span class="k">class</span> <span class="nc">Numbers</span><span class="p">(</span><span class="n">Enum</span><span class="p">):</span><span class="w">
</span> <span class="n">ONE</span> <span class="o">=</span> <span class="n">auto</span><span class="p">()</span><span class="w">
</span> <span class="n">TWO</span> <span class="o">=</span> <span class="n">auto</span><span class="p">()</span><span class="w">
</span> <span class="n">THREE</span> <span class="o">=</span> <span class="n">auto</span><span class="p">()</span><span class="w">
</span><span class="n">info</span> <span class="o">=</span> <span class="n">EnumInfo</span><span class="o">.</span><span class="n">fetch</span><span class="p">(</span><span class="n">conn</span><span class="p">,</span> <span class="s2">&quot;numbers&quot;</span><span class="p">)</span><span class="w">
</span><span class="n">register_enum</span><span class="p">(</span><span class="n">info</span><span class="p">,</span> <span class="n">conn</span><span class="p">,</span> <span class="n">Numbers</span><span class="p">)</span><span class="w">
</span><span class="n">conn</span><span class="o">.</span><span class="n">execute</span><span class="p">(</span><span class="s2">&quot;SELECT 'TWO'::numbers&quot;</span><span class="p">)</span><span class="o">.</span><span class="n">fetchone</span><span class="p">()[</span><span class="mi">0</span><span class="p">]</span><span class="w">
</span><span class="o">&lt;</span><span class="n">Numbers</span><span class="o">.</span><span class="n">TWO</span><span class="p">:</span> <span class="mi">2</span><span class="o">&gt;</span><span class="w">
</span><span class="n">conn</span><span class="o">.</span><span class="n">execute</span><span class="p">(</span><span class="s2">&quot;SELECT pg_typeof(</span><span class="si">%s</span><span class="s2">)&quot;</span><span class="p">,</span> <span class="p">[</span><span class="n">Numbers</span><span class="o">.</span><span class="n">ONE</span><span class="p">])</span><span class="o">.</span><span class="n">fetchone</span><span class="p">()[</span><span class="mi">0</span><span class="p">]</span><span class="w">
</span><span class="s1">'numbers'</span>
</pre>
<p>The facility can also be customized in order to adapt enums when the mapping
is not one-to-one:</p>
<pre class="code python literal-block">
<span class="k">class</span> <span class="nc">NumbersPlus</span><span class="p">(</span><span class="n">Enum</span><span class="p">):</span><span class="w">
</span> <span class="n">ONE</span> <span class="o">=</span> <span class="n">auto</span><span class="p">()</span><span class="w">
</span> <span class="n">TWO</span> <span class="o">=</span> <span class="n">auto</span><span class="p">()</span><span class="w">
</span> <span class="n">THREE</span> <span class="o">=</span> <span class="n">auto</span><span class="p">()</span><span class="w">
</span> <span class="n">THREE_PLUS</span> <span class="o">=</span> <span class="n">auto</span><span class="p">()</span> <span class="c1"># has some meaning in the program, but it is not stored</span><span class="w">
</span><span class="n">register_enum</span><span class="p">(</span><span class="n">info</span><span class="p">,</span> <span class="n">conn</span><span class="p">,</span> <span class="n">NumbersPlus</span><span class="p">,</span><span class="w">
</span> <span class="c1"># - Items not mentioned map naturally</span><span class="w">
</span> <span class="c1"># - This order gives THREE priority over THREE_PLUS when loading from db.</span><span class="w">
</span> <span class="n">mapping</span><span class="o">=</span><span class="p">[(</span><span class="n">NumbersPlus</span><span class="o">.</span><span class="n">THREE_PLUS</span><span class="p">,</span> <span class="s2">&quot;THREE&quot;</span><span class="p">),</span> <span class="p">(</span><span class="n">NumbersPlus</span><span class="o">.</span><span class="n">THREE</span><span class="p">,</span> <span class="s2">&quot;THREE&quot;</span><span class="p">)])</span><span class="w">
</span><span class="n">conn</span><span class="o">.</span><span class="n">execute</span><span class="p">(</span><span class="s2">&quot;SELECT </span><span class="si">%s</span><span class="s2">::text&quot;</span><span class="p">,</span><span class="w">
</span> <span class="p">[[</span><span class="n">NumbersPlus</span><span class="o">.</span><span class="n">ONE</span><span class="p">,</span> <span class="n">NumbersPlus</span><span class="o">.</span><span class="n">THREE_PLUS</span><span class="p">]]</span><span class="w">
</span><span class="p">)</span><span class="o">.</span><span class="n">fetchone</span><span class="p">()[</span><span class="mi">0</span><span class="p">]</span><span class="w">
</span><span class="s1">'{ONE,THREE}'</span><span class="w">
</span><span class="n">conn</span><span class="o">.</span><span class="n">execute</span><span class="p">(</span><span class="s2">&quot;select '{TWO,THREE}'::numbers[]&quot;</span><span class="p">)</span><span class="o">.</span><span class="n">fetchone</span><span class="p">()[</span><span class="mi">0</span><span class="p">]</span><span class="w">
</span><span class="p">[</span><span class="o">&lt;</span><span class="n">NumbersPlus</span><span class="o">.</span><span class="n">TWO</span><span class="p">:</span> <span class="mi">2</span><span class="o">&gt;</span><span class="p">,</span> <span class="o">&lt;</span><span class="n">NumbersPlus</span><span class="o">.</span><span class="n">THREE</span><span class="p">:</span> <span class="mi">3</span><span class="o">&gt;</span><span class="p">]</span>
</pre>
</div>
<div class="section" id="in-partnership-with-cockroachdb">
<h2>In partnership with CockroachDB</h2>
<p><a class="reference external" href="https://www.cockroachlabs.com/">CockroachDB</a> is a distributed database
presenting an SQL interface on top of a distributed key-value store. Although
it is a completely independent implementation, it uses the same PostgreSQL
client-server protocol.</p>
<p>In the past few months, we have collaborated to create an even smoother
integration, so that every PostgreSQL feature, also supported by CockroachDB,
can be used in a transparent way: <a class="reference external" href="https://www.psycopg.org/psycopg3/docs/advanced/cursors.html#server-side-cursors">server-side cursors</a>,
<a class="reference external" href="https://www.psycopg.org/psycopg3/docs/advanced/pipeline.html">pipeline mode</a>, CockroachDB
data types are all supported out-of-the-box.</p>
<p>CockroachDB also implements <a class="reference external" href="https://www.cockroachlabs.com/docs/stable/changefeed-for.html">CHANGEFEED</a>, a
streaming query, which Psycopg can consume using its Cursor.stream() feature.
This immediately receives every change happening in a database table, enabling
interesting new ways to write distributed applications.</p>
</div>
<div class="section" id="executemany-improvements">
<h2><tt class="docutils literal">executemany()</tt> improvements</h2>
<p>The <a class="reference external" href="https://www.psycopg.org/psycopg3/docs/api/cursors.html#psycopg.Cursor.executemany">executemany()</a>
method now supports returning the executed statements' output. You can now for
instance execute an <tt class="docutils literal">INSERT ... RETURNING</tt> on a batch of records and
retrieve the ids associated to the newly inserted records.</p>
<p><tt class="docutils literal">executemany()</tt> automatically uses <a class="reference external" href="https://www.psycopg.org/psycopg3/docs/advanced/pipeline.html">the pipeline mode</a> already
described, making use of pipeline mode speedups without changing any code in
the programs using this method.</p>
</div>
<div class="section" id="and-many-more-improvements">
<h2>And many more improvements</h2>
<ul class="simple">
<li>The <a class="reference external" href="https://www.psycopg.org/psycopg3/docs/basic/transactions.html#two-phase-commit">Two-phase commit protocol</a>
is now available as per <a class="reference external" href="https://peps.python.org/pep-0249/#optional-two-phase-commit-extensions">DBAPI specification</a>.</li>
<li>Asynchronous connections <a class="reference external" href="https://www.psycopg.org/psycopg3/docs/advanced/async.html#asynchronous-operations">don't block on DNS names resolution on connect</a>.</li>
<li><tt class="docutils literal">Cursor.copy()</tt> now <a class="reference external" href="https://www.psycopg.org/psycopg3/docs/api/cursors.html#psycopg.Cursor.copy">takes parameters</a>,
like a normal query.</li>
<li>It is also possible to <a class="reference external" href="https://www.psycopg.org/psycopg3/docs/api/copy.html#copy-writers">replace the writer of a 'Copy' object</a>, in
order to use psycopg just to format data in COPY format and to do something
else with the data produced, for instance save it to a file for later
processing.</li>
<li>...And many more improvements you can find in our <a class="reference external" href="https://www.psycopg.org/psycopg3/docs/news.html#psycopg-3-1">release notes</a>.</li>
</ul>
</div>
<div class="section" id="thank-you-very-much">
<h2>Thank you very much!</h2>
<p>We hope you will enjoy to use Psycopg 3.1 and will benefit from its new
features. Psycopg 3 is developed and maintained thanks to the support
of <a class="reference external" href="https://www.psycopg.org/sponsors/">our sponsors</a>.</p>
<p>If you are a Python and PostgreSQL user and would like to make sure that the
interface between the two is well maintained and continuously improved, please
consider <a class="reference external" href="https://github.com/sponsors/dvarrazzo">sponsoring the project 💜</a>.</p>
<p>Thank you very much, happy hacking!</p>
</div>
</content></entry><entry><title>Psycopg 3.0 released</title><link href="https://www.psycopg.org/articles/2021/10/13/psycopg-30-released/" rel="alternate"></link><updated>2021-10-13T00:00:00Z</updated><author><name>Daniele Varrazzo</name></author><id>urn:uuid:060c020e-db6c-391f-b8bc-af629192c059</id><content type="html"><p>Hello,</p>
<p>I am extremely excited to announce the first stable release of Psycopg 3!</p>
<!-- CUT-HERE -->
<p>Psycopg 3 is a complete rewrite based on the experience accumulated with the
development and maintenance of psycopg2. Psycopg 3 targets all the current
versions of Python (3.6-3.10) and PostgreSQL (10-14) and allows the use of
modern Python development techniques, such as async and statically typed code.
<a class="reference external" href="https://www.psycopg.org/psycopg3/docs/">A list of the new features</a> is
available in the documentation.</p>
<p>This was a long journey: I would like to thank the people who have helped to
make this project amazing with their ideas and their code: Denis Laxalde (row
factories), Daniel Fortunov (transaction blocks), Jacopo Farina (PostGIS
support) and many who have tested, helped, discussed, cheered for us.</p>
<p>And an immense thank you to the sponsors who have made this project possible:
<a class="reference external" href="https://postgrespro.com/">Postgres Professional</a> and <a class="reference external" href="https://www.commandprompt.com/">Command Prompt</a> have been our biggest supporters so far,
but <a class="reference external" href="https://www.psycopg.org/sponsors/">many companies and individuals</a>
have given their generous contribution. Surely there will be more work to come
in the future: if you want you can help <a class="reference external" href="https://github.com/sponsors/dvarrazzo/">sponsoring the project</a> too.</p>
<p>If you would like to try out the project please check out the <a class="reference external" href="https://www.psycopg.org/psycopg3/docs/basic/install.html">installation
and usage instructions</a>. We are eager to
hear your feedback and to share this journey with you.</p>
<p>--- Daniele, on behalf of Psycopg</p>
</content></entry><entry><title>Psycopg 3.0 beta 1 released!</title><link href="https://www.psycopg.org/articles/2021/08/30/psycopg-30-beta1-released/" rel="alternate"></link><updated>2021-08-30T00:00:00Z</updated><author><name>Daniele Varrazzo</name></author><id>urn:uuid:e099f1fe-54e1-3c67-849f-2c3d0d1ecb27</id><content type="html"><p>We are immensely proud to <a class="reference external" href="https://pypi.org/project/psycopg/3.0b1/">release on PyPI</a> the first beta package of Psycopg
3!</p>
<!-- CUT-HERE -->
<p>Psycopg 3 is a complete rewrite of Psycopg 2, maintaining the same fundamental
libpq wrapper architecture and DB-API interface design, but exposing new
features to better work with the newer versions of Python and PostgreSQL.</p>
<p>On the Python side, Psycopg 3 allows the use of asyncio-based concurrency and
static typing. Many improvement to the Python interface make the library much
simpler and more idiomatic to use,</p>
<p>On the PostgreSQL side, Psycopg 3 makes use of server-side parameters,
prepared statements, binary parameters, and great support for COPY operations.</p>
<p>But the most outstanding feature of the project is not a technical one:
Psycopg 3 was made possible by the great generosity of many <a class="reference external" href="https://www.psycopg.org/sponsors/">sponsors</a>, who have funded the development of the
project. Among the many backers, we are especially grateful to <a class="reference external" href="https://postgrespro.com/">Postgres
Professional</a> and <a class="reference external" href="https://www.commandprompt.com/">Command Prompt, Inc</a>, which have given the most outstanding
support. But many other companies and individuals, each one in their capacity,
have shown concrete support for free software development and progress. We
sincerely hope that you will find this work useful and that you will feel
proud for having contributed to it.</p>
<p><a class="reference external" href="https://www.psycopg.org/sponsors/">https://www.psycopg.org/sponsors/</a></p>
<p>Where do we go from here? The hope is that the interface of the adapter will
not change excessively before a definitive 3.0 release: the project has
already been used in a few production environments, in the past months, and a
lot of real world feedback has already helped to improve the interface and
functionalities. We invite you to test the project and give us your feedback.
So...</p>
<pre class="literal-block">
pip install -U pip
pip install --pre psycopg[binary]
</pre>
<p>Please try it, test it, and let us know how it goes!</p>
<p>For more info you can dive into the docs: start from the <a class="reference external" href="https://www.psycopg.org/psycopg3/docs/basic/install.html">install</a> and <a class="reference external" href="https://www.psycopg.org/psycopg3/docs/basic/usage.html">usage</a>
pages, I'm sure you will find your way.</p>
<p>Happy hacking!</p>
</content></entry><entry><title>Psycopg 2.9 released</title><link href="https://www.psycopg.org/articles/2021/06/16/psycopg-29-released/" rel="alternate"></link><updated>2021-06-16T00:00:00Z</updated><author><name>Daniele Varrazzo</name></author><id>urn:uuid:5ffbbc8a-c1c5-32ed-bdf5-6eac4b92e794</id><content type="html"><p>Psycopg 2.9 has been released!</p>
<p>This is a relatively small release compared to previous major releases.
However the creation of the packages took a lot of effort. The previously used
CI system now has reduced support for free software projects - it was decided
that package building should be moved to GitHub Actions.</p>
<p>Packaging has also become more complex because of the evolution of the Python
packaging standards and the need to support multiple architectures (Intel,
ARM, PPC...).</p>
<p>Maintaining a project such as Psycopg requires a lot of effort. For this
reason, we are extremely grateful to all our <a class="reference external" href="https://www.psycopg.org/sponsors/">sponsors</a> who are enabling the
maintenance and development of Psycopg. Thank you very much! <span class="raw-html"><i
class="fa fa-heart"></i></span></p>
<!-- CUT-HERE -->
<p>What’s new in psycopg 2.9</p>
<ul class="simple">
<li><tt class="docutils literal">with connection</tt> starts a transaction on autocommit transactions too (<a class="reference external" href="https://github.com/psycopg/psycopg2/issues/941">ticket #941</a>).</li>
<li>Timezones with fractional minutes are supported on Python 3.7 and following (<a class="reference external" href="https://github.com/psycopg/psycopg2/issues/1272">ticket #1272</a>).</li>
<li>Escape table and column names in <tt class="docutils literal">copy_from()</tt> and <tt class="docutils literal">copy_to()</tt>.</li>
<li>Connection exceptions with sqlstate 08XXX reclassified as <tt class="docutils literal">OperationalError</tt> (a subclass of the previously used <tt class="docutils literal">DatabaseError</tt>) (<a class="reference external" href="https://github.com/psycopg/psycopg2/issues/1148">ticket #1148</a>).</li>
<li>Include library dirs required from libpq to work around MacOS build problems (<a class="reference external" href="https://github.com/psycopg/psycopg2/issues/1200">ticket #1200</a>).</li>
</ul>
<p>Other changes:</p>
<ul class="simple">
<li>Dropped support for Python 2.7, 3.4, 3.5 (<a class="reference external" href="https://github.com/psycopg/psycopg2/issues/1198">ticket #1198</a>, <a class="reference external" href="https://github.com/psycopg/psycopg2/issues/1000">ticket #1000</a>, <a class="reference external" href="https://github.com/psycopg/psycopg2/issues/1197">ticket #1197</a>).</li>
<li>Dropped support for <tt class="docutils literal">mx.DateTime</tt>.</li>
<li>Use <tt class="docutils literal">datetime.timezone</tt> objects by default in <tt class="docutils literal">datetime</tt> objects instead of <tt class="docutils literal">FixedOffsetTimezone</tt>.</li>
<li>The <tt class="docutils literal">psycopg2.tz</tt> module is deprecated and scheduled to be dropped in the next major release.</li>
<li>Provide <a class="reference external" href="https://peps.python.org/pep-0599">PEP 599</a> wheels packages (<tt class="docutils literal">manylinux2014</tt> tag) for <tt class="docutils literal">i686</tt> and <tt class="docutils literal">x86_64</tt> platforms.</li>
<li>Provide <a class="reference external" href="https://peps.python.org/pep-0600">PEP 600</a> wheels packages (<tt class="docutils literal">manylinux_2_24</tt> tag) for <tt class="docutils literal">aarch64</tt> and <tt class="docutils literal">ppc64le</tt> platforms.</li>
<li>Wheel package compiled against OpenSSL 1.1.1k and PostgreSQL 13.3.</li>
<li>Build system for Linux/MacOS binary packages moved to GitHub Actions.</li>
</ul>
</content></entry><entry><title>The psycopg3 adaptation system</title><link href="https://www.psycopg.org/articles/2020/11/24/psycopg3-adaptation/" rel="alternate"></link><updated>2020-11-24T00:00:00Z</updated><author><name>Daniele Varrazzo</name></author><id>urn:uuid:eef60882-aee1-318d-9e8b-771bfe1d5026</id><content type="html"><p>The adaptation system between Python objects and PostgreSQL types is at the
core of <tt class="docutils literal">psycopg2</tt> and <tt class="docutils literal">psycopg3</tt>. The flexibility of the <tt class="docutils literal">psycopg2</tt>
adaptation system provides <a class="reference external" href="https://www.psycopg.org/docs/usage.html#adaptation-of-python-values-to-sql-types">good out-of-the-box object mapping</a> and allows
users to <a class="reference external" href="https://www.psycopg.org/docs/advanced.html#adapting-new-python-types-to-sql-syntax">customise it to suit any need</a>. Do you want your <tt class="docutils literal">decimal</tt>
numbers returned as <tt class="docutils literal">float</tt> because you need speed over pennies? Do you want
to map PostgreSQL Infinity dates to the 25th of December 3099? That's
certainly doable.</p>
<blockquote>
</blockquote>
<p>The <tt class="docutils literal">psycopg3</tt> adaptation system needs some modification compared to
<tt class="docutils literal">psycopg2</tt>, because <tt class="docutils literal">psycopg3</tt> uses the &quot;extended query protocol&quot; to send
query parameters separately from the query. Together, with the differences to
accommodate, there is also a chance to improve a system that has been in use
for several years and has shown its shortcomings together with its strengths.</p>
<!-- CUT-HERE -->
<div class="section" id="server-side-binding">
<h2>Server-side binding</h2>
<p>Server-side parameter binding has been a long-time desired feature. So far
<tt class="docutils literal">psycopg2</tt> has adapted arguments on the client-side:</p>
<pre class="code python literal-block">
<span class="c1"># psycopg2</span><span class="w">
</span><span class="n">cursor</span><span class="o">.</span><span class="n">execute</span><span class="p">(</span><span class="w">
</span> <span class="s2">&quot;INSERT INTO tbl (s, n, d) VALUES (</span><span class="si">%s</span><span class="s2">, </span><span class="si">%s</span><span class="s2">, </span><span class="si">%s</span><span class="s2">)&quot;</span><span class="p">,</span><span class="w">
</span> <span class="p">[</span><span class="mi">42</span><span class="p">,</span> <span class="s2">&quot;Hel'lo&quot;</span><span class="p">,</span> <span class="n">date</span><span class="p">(</span><span class="mi">2020</span><span class="p">,</span> <span class="mi">12</span><span class="p">,</span> <span class="mi">31</span><span class="p">)])</span><span class="w">
</span><span class="c1"># has arguments adapted and quoted:</span><span class="w">
</span><span class="n">args</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="o">=</span> <span class="s2">&quot;42&quot;</span><span class="w">
</span><span class="n">args</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span> <span class="o">=</span> <span class="s2">&quot;'Hel''lo'&quot;</span><span class="w">
</span><span class="n">args</span><span class="p">[</span><span class="mi">2</span><span class="p">]</span> <span class="o">=</span> <span class="s2">&quot;'2020-12-31'::date&quot;</span><span class="w">
</span><span class="c1"># merged to the query:</span><span class="w">
</span><span class="n">query</span> <span class="o">=</span> <span class="n">query</span> <span class="o">%</span> <span class="n">args</span><span class="w">
</span><span class="c1"># and passed as a single string:</span><span class="w">
</span><span class="n">libpq</span><span class="o">.</span><span class="n">PQexec</span><span class="p">(</span><span class="w">
</span> <span class="s2">&quot;INSERT INTO tbl (s, n, d) VALUES (42, 'Hel''lo', '2020-12-31'::date)&quot;</span><span class="p">)</span>
</pre>
<p>The <tt class="docutils literal">psycopg3</tt> end user interface is unchanged, but behind the scenes will only
perform adaptation, not quoting, and will send the arguments separately:</p>
<pre class="code python literal-block">
<span class="c1"># psycopg3</span><span class="w">
</span><span class="n">cursor</span><span class="o">.</span><span class="n">execute</span><span class="p">(</span><span class="w">
</span> <span class="s2">&quot;INSERT INTO tbl (s, n, d) VALUES (</span><span class="si">%s</span><span class="s2">, </span><span class="si">%s</span><span class="s2">, </span><span class="si">%s</span><span class="s2">)&quot;</span><span class="p">,</span><span class="w">
</span> <span class="p">[</span><span class="mi">42</span><span class="p">,</span> <span class="s2">&quot;Hel'lo&quot;</span><span class="p">,</span> <span class="n">date</span><span class="p">(</span><span class="mi">2020</span><span class="p">,</span> <span class="mi">12</span><span class="p">,</span> <span class="mi">31</span><span class="p">)])</span><span class="w">
</span><span class="c1"># has arguments adapted:</span><span class="w">
</span><span class="n">args</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="o">=</span> <span class="s2">&quot;42&quot;</span><span class="w">
</span><span class="n">args</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span> <span class="o">=</span> <span class="s2">&quot;Hel'lo&quot;</span><span class="w">
</span><span class="n">args</span><span class="p">[</span><span class="mi">2</span><span class="p">]</span> <span class="o">=</span> <span class="s2">&quot;2020-12-31&quot;</span><span class="w">
</span><span class="c1"># and passed as separate information:</span><span class="w">
</span><span class="n">libpq</span><span class="o">.</span><span class="n">PQexecParams</span><span class="p">(</span><span class="w">
</span> <span class="s2">&quot;INSERT INTO tbl (s, n, d) VALUES ($1, $2, $3)&quot;</span><span class="p">,</span> <span class="c1"># Postgres placeholders</span><span class="w">
</span> <span class="p">[</span><span class="s2">&quot;42&quot;</span><span class="p">,</span> <span class="s2">&quot;Hel'lo&quot;</span><span class="p">,</span> <span class="s2">&quot;2020-12-31&quot;</span><span class="p">],</span> <span class="c1"># Postgres formats, no quoting</span><span class="w">
</span> <span class="n">oids</span><span class="o">=</span><span class="p">[</span><span class="n">INT8_OID</span><span class="p">,</span> <span class="mi">0</span><span class="p">,</span> <span class="n">DATE_OID</span><span class="p">])</span> <span class="c1"># Type indications if available</span>
</pre>
<p>Server-side binding brings better performance, the possibility to use prepared
statements and binary data, as well as better integration with server-side
logging and monitoring. Theoretically it also brings better safety against SQL
injections, but <tt class="docutils literal">psycopg2</tt> already does a good job at <a class="reference external" href="https://www.psycopg.org/docs/usage.html#the-problem-with-the-query-parameters">providing a safe path
for parameter binding</a>: in <tt class="docutils literal">psycopg2</tt>, creating an unsafe query is already
harder than doing things the right way.</p>
<p>However server-side binding brings a few incompatibilities, such as:</p>
<ul>
<li><p class="first">cannot send more than one query at once if parameters are used:</p>
<pre class="code python literal-block">
<span class="c1"># Must use separate queries or psycopg3.sql client-side adaptation</span><span class="w">
</span><span class="n">cur</span><span class="o">.</span><span class="n">execute</span><span class="p">(</span><span class="w">
</span> <span class="s2">&quot;INSERT INTO tbl1 VALUES (</span><span class="si">%s</span><span class="s2">); INSERT INTO tbl2 VALUES (</span><span class="si">%s</span><span class="s2">)&quot;</span><span class="p">,</span><span class="w">
</span> <span class="p">[</span><span class="mi">10</span><span class="p">,</span> <span class="mi">20</span><span class="p">])</span>
</pre>
</li>
<li><p class="first">cannot use certain commands such as <tt class="docutils literal">SET</tt> or <tt class="docutils literal">NOTIFY</tt> with parameters:</p>
<pre class="code python literal-block">
<span class="c1"># Must use &quot;SELECT set_config('timezone', %s)&quot;</span><span class="w">
</span><span class="n">cur</span><span class="o">.</span><span class="n">execute</span><span class="p">(</span><span class="s2">&quot;SET timezone to </span><span class="si">%s</span><span class="s2">&quot;</span><span class="p">,</span> <span class="p">[</span><span class="n">timezone</span><span class="p">])</span><span class="w">
</span><span class="c1"># Must use &quot;SELECT pg_notify('channel', %s)&quot;</span><span class="w">
</span><span class="n">cur</span><span class="o">.</span><span class="n">execute</span><span class="p">(</span><span class="s2">&quot;NOTIFY channel, </span><span class="si">%s</span><span class="s2">&quot;</span><span class="p">,</span> <span class="p">[</span><span class="n">message</span><span class="p">])</span>
</pre>
</li>
<li><p class="first">cannot use the <tt class="docutils literal">IN <span class="pre">(...)</span></tt> construct:</p>
<pre class="code python literal-block">
<span class="c1"># Must use &quot;AND nation = any (%s)&quot;</span><span class="w">
</span><span class="n">cur</span><span class="o">.</span><span class="n">execute</span><span class="p">(</span><span class="s2">&quot;... AND nation in </span><span class="si">%s</span><span class="s2">&quot;</span><span class="p">,</span> <span class="p">[(</span><span class="s2">&quot;IT&quot;</span><span class="p">,</span> <span class="s2">&quot;FR&quot;</span><span class="p">,</span> <span class="s2">&quot;DE&quot;</span><span class="p">)])</span>
</pre>
</li>
</ul>
<p>All in all, most queries will just work, however the few incompatibilities
require a non-backward-compatible change in version.</p>
</div>
<div class="section" id="a-new-adaptation-system">
<h2>A new adaptation system</h2>
<p>Embracing the new type of communication requires to change the way Python
parameters and PostgreSQL data types are adapted. The new system:</p>
<ul>
<li><p class="first">cannot use the whole SQL syntax, but must limit to literals. For instance
the Python list <tt class="docutils literal">[&quot;foo, bar&quot;, &quot;Hel'lo&quot;]</tt> cannot be expressed with
<tt class="docutils literal"><span class="pre">ARRAY['foo,</span> bar', <span class="pre">'Hel''lo']</span></tt> but must use array literal rules and become
<tt class="docutils literal">{&quot;foo, <span class="pre">bar&quot;,Hel'lo}</span></tt>.</p>
</li>
<li><p class="first">Makes possible to specify type OIDs, in case that's useful (<a class="reference external" href="https://www.varrazzo.com/blog/2020/11/07/psycopg3-adaptation/">not always an
easy choice</a>).</p>
</li>
<li><p class="first">Allows for the use of binary types, which is especially useful for a large
binary blob, otherwise things become bloated by <a class="reference external" href="https://www.postgresql.org/docs/current/datatype-binary.html">binary escaping</a> and have
to traverse the many layers of lexing/parsing in the server, each with its
own memory copy. This can be done by simply using the <tt class="docutils literal">%b</tt> placeholder
over <tt class="docutils literal">%s</tt>:</p>
<pre class="code python literal-block">
<span class="k">with</span> <span class="nb">open</span><span class="p">(</span><span class="n">image_name</span><span class="p">,</span> <span class="s2">&quot;rb&quot;</span><span class="p">)</span> <span class="k">as</span> <span class="n">f</span><span class="p">:</span><span class="w">
</span> <span class="n">cur</span><span class="o">.</span><span class="n">execute</span><span class="p">(</span><span class="w">
</span> <span class="s2">&quot;INSERT INTO images (name, data) VALUES (</span><span class="si">%s</span><span class="s2">, %b)&quot;</span><span class="p">,</span><span class="w">
</span> <span class="p">[</span><span class="n">image_name</span><span class="p">,</span> <span class="n">f</span><span class="o">.</span><span class="n">read</span><span class="p">()])</span>
</pre>
</li>
</ul>
<p>It's also a good chance to review the work that must be done by the client to
adapt values. <tt class="docutils literal">psycopg2</tt> creates several instances of &quot;adapter&quot; wrappers,
one for each value adapted. In <tt class="docutils literal">psycopg3</tt>, the adaptation objects have a
different life cycle: choices based on the environment, such as the connection
encoding, can be made once for each Python type, rather than once per value,
doing radically less work for each converted object. You can check out the
<a class="reference external" href="https://www.psycopg.org/psycopg3/docs/advanced/adapt.html">psycopg3 adaptation documentation</a> for all the details.</p>
</div>
<div class="section" id="customising-psycopg3-adaptation">
<h2>Customising psycopg3 adaptation</h2>
<p>Customising types adaptation can now be done using <a class="reference external" href="https://www.psycopg.org/psycopg3/docs/advanced/adapt.html#psycopg3.adapt.Dumper">Dumper</a> classes, either
creating new ones or mapping existing ones on different Python classes to save
to the database. For instance, the builtin <tt class="docutils literal">DateDumper</tt> converts Python
dates to PostgreSQL ones. PostgreSQL can handle an &quot;infinity&quot; date, which
Python cannot. If we wanted to store Python's <tt class="docutils literal">date.max</tt> to PostgreSQL
infinity, we could create a subclass for the dumper and register it in the
scope we want to use it, globally or just on a connection or cursor:</p>
<pre class="code python literal-block">
<span class="k">class</span> <span class="nc">InfDateDumper</span><span class="p">(</span><span class="n">DateDumper</span><span class="p">):</span><span class="w">
</span> <span class="k">def</span> <span class="nf">dump</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">obj</span><span class="p">):</span><span class="w">
</span> <span class="k">if</span> <span class="n">obj</span> <span class="o">==</span> <span class="n">date</span><span class="o">.</span><span class="n">max</span><span class="p">:</span><span class="w">
</span> <span class="k">return</span> <span class="sa">b</span><span class="s2">&quot;infinity&quot;</span><span class="w">
</span> <span class="k">else</span><span class="p">:</span><span class="w">
</span> <span class="k">return</span> <span class="nb">super</span><span class="p">()</span><span class="o">.</span><span class="n">dump</span><span class="p">(</span><span class="n">obj</span><span class="p">)</span><span class="w">
</span><span class="n">cur</span><span class="o">.</span><span class="n">execute</span><span class="p">(</span><span class="s2">&quot;SELECT </span><span class="si">%s</span><span class="s2">::text, </span><span class="si">%s</span><span class="s2">::text&quot;</span><span class="p">,</span> <span class="p">[</span><span class="n">date</span><span class="p">(</span><span class="mi">2020</span><span class="p">,</span> <span class="mi">12</span><span class="p">,</span> <span class="mi">31</span><span class="p">),</span> <span class="n">date</span><span class="o">.</span><span class="n">max</span><span class="p">])</span><span class="o">.</span><span class="n">fetchone</span><span class="p">()</span><span class="w">
</span><span class="c1"># ('2020-12-31', '9999-12-31')</span><span class="w">
</span><span class="n">InfDateDumper</span><span class="o">.</span><span class="n">register</span><span class="p">(</span><span class="n">date</span><span class="p">,</span> <span class="n">cur</span><span class="p">)</span><span class="w">
</span><span class="n">cur</span><span class="o">.</span><span class="n">execute</span><span class="p">(</span><span class="s2">&quot;SELECT </span><span class="si">%s</span><span class="s2">::text, </span><span class="si">%s</span><span class="s2">::text&quot;</span><span class="p">,</span> <span class="p">[</span><span class="n">date</span><span class="p">(</span><span class="mi">2020</span><span class="p">,</span> <span class="mi">12</span><span class="p">,</span> <span class="mi">31</span><span class="p">),</span> <span class="n">date</span><span class="o">.</span><span class="n">max</span><span class="p">])</span><span class="o">.</span><span class="n">fetchone</span><span class="p">()</span><span class="w">
</span><span class="c1"># ('2020-12-31', 'infinity')</span>
</pre>
<p>The system is pretty symmetric and employs similar <a class="reference external" href="https://www.psycopg.org/psycopg3/docs/advanced/adapt.html#psycopg3.adapt.Loader">Loader</a> objects to map OIDs
to the code responsible for its decoding. For instance, if we wanted to
reverse the above customisation and map PostgreSQL infinity date to
<tt class="docutils literal">date.max</tt> (instead of raising an exception), it could be done using a
subclass of the builtin loader (or using an entirely new object if required):</p>
<pre class="code python literal-block">
<span class="k">class</span> <span class="nc">InfDateLoader</span><span class="p">(</span><span class="n">DateLoader</span><span class="p">):</span><span class="w">
</span> <span class="k">def</span> <span class="nf">load</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">data</span><span class="p">):</span><span class="w">
</span> <span class="k">if</span> <span class="n">data</span> <span class="o">==</span> <span class="sa">b</span><span class="s2">&quot;infinity&quot;</span><span class="p">:</span><span class="w">
</span> <span class="k">return</span> <span class="n">date</span><span class="o">.</span><span class="n">max</span><span class="w">
</span> <span class="k">else</span><span class="p">:</span><span class="w">
</span> <span class="k">return</span> <span class="nb">super</span><span class="p">()</span><span class="o">.</span><span class="n">load</span><span class="p">(</span><span class="n">data</span><span class="p">)</span><span class="w">
</span><span class="n">cur</span><span class="o">.</span><span class="n">execute</span><span class="p">(</span><span class="s2">&quot;select '2020-12-31'::date, 'infinity'::date&quot;</span><span class="p">)</span><span class="o">.</span><span class="n">fetchone</span><span class="p">()</span><span class="w">
</span><span class="c1"># Raises DataError: Python date doesn't support years after 9999: got infinity</span><span class="w">
</span><span class="kn">from</span> <span class="nn">psycopg3.oids</span> <span class="kn">import</span> <span class="n">builtins</span><span class="w">
</span><span class="n">InfDateLoader</span><span class="o">.</span><span class="n">register</span><span class="p">(</span><span class="n">builtins</span><span class="p">[</span><span class="s2">&quot;date&quot;</span><span class="p">]</span><span class="o">.</span><span class="n">oid</span><span class="p">,</span> <span class="n">cur</span><span class="p">)</span><span class="w">
</span><span class="n">cur</span><span class="o">.</span><span class="n">execute</span><span class="p">(</span><span class="s2">&quot;select '2020-12-31'::date, 'infinity'::date&quot;</span><span class="p">)</span><span class="o">.</span><span class="n">fetchone</span><span class="p">()</span><span class="w">
</span><span class="p">(</span><span class="n">datetime</span><span class="o">.</span><span class="n">date</span><span class="p">(</span><span class="mi">2020</span><span class="p">,</span> <span class="mi">12</span><span class="p">,</span> <span class="mi">31</span><span class="p">),</span> <span class="n">datetime</span><span class="o">.</span><span class="n">date</span><span class="p">(</span><span class="mi">9999</span><span class="p">,</span> <span class="mi">12</span><span class="p">,</span> <span class="mi">31</span><span class="p">))</span>
</pre>
<p>The customisation automatically applies to recursive types, such as arrays or
composite types: if the date loader is customised then the date array works as
expected:</p>
<pre class="code python literal-block">
<span class="n">cur</span><span class="o">.</span><span class="n">execute</span><span class="p">(</span><span class="s2">&quot;select '{2020-12-31,infinity}'::date[]&quot;</span><span class="p">)</span><span class="o">.</span><span class="n">fetchone</span><span class="p">()</span><span class="w">
</span><span class="p">([</span><span class="n">datetime</span><span class="o">.</span><span class="n">date</span><span class="p">(</span><span class="mi">2020</span><span class="p">,</span> <span class="mi">12</span><span class="p">,</span> <span class="mi">31</span><span class="p">),</span> <span class="n">datetime</span><span class="o">.</span><span class="n">date</span><span class="p">(</span><span class="mi">9999</span><span class="p">,</span> <span class="mi">12</span><span class="p">,</span> <span class="mi">31</span><span class="p">)],)</span>
</pre>
<p>All in all the new adaptation system provides better performance and easier
customisation <a class="reference external" href="https://www.psycopg.org/docs/advanced.html#adapting-new-python-types-to-sql-syntax">compared to psycopg2</a>. The new adapters are easier to
compose, such as using them in <a class="reference external" href="/articles/2020/11/15/psycopg3-copy/">COPY operations</a>. And if client-side adaptation is still needed (to
generate dynamically data definition statements, to prepare offline update
scripts...) <a class="reference external" href="https://www.psycopg.org/psycopg3/docs/sql.html">psycopg3.sql</a> still allows for the flexibility of client-side
query composition.</p>
</div>
<div class="section" id="project-sponsorship">
<h2>Project sponsorship</h2>
<div class="sponsor docutils container">
<p>The new adaptation system is one of several new features that are being
designed and implemented in <a class="reference external" href="/psycopg3/">psycopg3</a>. The project is currently <a class="reference external" href="https://github.com/psycopg/psycopg3">under
active development</a>: if there is enough support it will be possible to
work at the project full-time and bring it to release swiftly.</p>
<p>If you use Python and PostgreSQL, and you would like to support the
creation of the most advanced adapter between the two systems, <a class="reference external" href="https://github.com/sponsors/dvarrazzo/">please
consider becoming a sponsor</a>. Thank you!</p>
<iframe src="https://github.com/sponsors/dvarrazzo/button"
title="Sponsor the project" style="border: 0"
width="116" height="35">
</iframe></div>
</div>
</content></entry><entry><title>The new COPY support in psycopg3</title><link href="https://www.psycopg.org/articles/2020/11/15/psycopg3-copy/" rel="alternate"></link><updated>2020-11-15T00:00:00Z</updated><author><name>Daniele Varrazzo</name></author><id>urn:uuid:67b93afe-21e3-30fb-8a32-4dbaf5daaa4e</id><content type="html"><p><tt class="docutils literal">psycopg2</tt> allows <a class="reference external" href="https://www.psycopg.org/docs/usage.html#copy">interaction with PostgreSQL COPY commands</a>. However
what is possible to do with them is relatively limited: the only possible
interaction is with file-like objects:</p>
<ul class="simple">
<li>there is no adaptation from Python objects to PostgreSQL, as there is for
normal queries: data must be formatted &quot;manually&quot; by the user;</li>
<li>psycopg2 &quot;pulls&quot; data from the file: writing a system that produces data and
pushes it into PostgreSQL is a very contrived operation, requiring to write
a blocking file-like object;</li>
<li>there is no support for <a class="reference external" href="https://github.com/psycopg/psycopg2/issues/428">asynchronous copy</a>.</li>
</ul>
<p><a class="reference external" href="/psycopg3/">psycopg3</a> <a class="reference external" href="https://www.psycopg.org/psycopg3/docs/basic/copy.html#copy">addresses these shortcomings</a> and makes it easy to write Python
programs producing data and pushing it efficiently to the database using the
<tt class="docutils literal">COPY</tt> protocol.</p>
<!-- CUT-HERE -->
<p><tt class="docutils literal">psycopg2</tt> adaptation system is designed to compose queries client-side, so
it is concerned with the right use of the quotes: the python string
<tt class="docutils literal">O'Reilly</tt> is converted to <tt class="docutils literal"><span class="pre">'O''Reilly'</span></tt>, and the <tt class="docutils literal">date(2020, 11, 15)</tt>
to <tt class="docutils literal"><span class="pre">'2020-11-15'::date</span></tt>. These extra quotes get in the way of COPY, and
there isn't an intermediate level where a conversion to string is performed,
but no quote or other SQL construct are added.</p>
<p><tt class="docutils literal">psycopg3</tt> uses the PostgreSQL extended query protocol and sends query and
parameters separately. Parameters require adaptation to the PostgreSQL
formats, but quoting, and quotes escaping, are no more its concern: the string
<tt class="docutils literal">O'Reilly</tt> doesn't need further manipulation and the date is converted only
to the string <tt class="docutils literal"><span class="pre">2020-11-15</span></tt>; types information are passed as additional
separate information according to <a class="reference external" href="https://www.postgresql.org/docs/current/libpq-exec.html#LIBPQ-PQEXECPARAMS">the libpq API</a>.</p>
<p>The server-side format of these values are exactly what the <tt class="docutils literal">COPY FROM</tt>
command expects, so it's now easy to compose a row by adapting Python objects
and to pass</p>
<pre class="literal-block">
O'Reilly\t2020-11-15\n
</pre>
<p>to the server. The mechanism to do so is exposed to Python by a new context
manager, returned by the <a class="reference external" href="https://www.psycopg.org/psycopg3/docs/api/cursors.html#psycopg.Cursor.copy">Cursor.copy()</a> method, which enables to write:</p>
<pre class="code python literal-block">
<span class="k">with</span> <span class="n">cursor</span><span class="o">.</span><span class="n">copy</span><span class="p">(</span><span class="s2">&quot;COPY mytable FROM STDIN&quot;</span><span class="p">)</span> <span class="k">as</span> <span class="n">copy</span><span class="p">:</span><span class="w">
</span> <span class="n">copy</span><span class="o">.</span><span class="n">write_row</span><span class="p">((</span><span class="s2">&quot;O'Reilly&quot;</span><span class="p">,</span> <span class="n">date</span><span class="p">(</span><span class="mi">2020</span><span class="p">,</span> <span class="mi">11</span><span class="p">,</span> <span class="mi">15</span><span class="p">)))</span>
</pre>
<p>Any list of tuples of values, or generator of sequences of values, can be used
to push data into Postgres:</p>
<pre class="code python literal-block">
<span class="k">with</span> <span class="n">cursor</span><span class="o">.</span><span class="n">copy</span><span class="p">(</span><span class="s2">&quot;COPY mytable FROM STDIN&quot;</span><span class="p">)</span> <span class="k">as</span> <span class="n">copy</span><span class="p">:</span><span class="w">
</span> <span class="k">for</span> <span class="n">record</span> <span class="ow">in</span> <span class="n">my_generator</span><span class="p">():</span><span class="w">
</span> <span class="n">copy</span><span class="o">.</span><span class="n">write_row</span><span class="p">(</span><span class="n">record</span><span class="p">)</span>
</pre>
<p>The copy operation is concluded as soon as the <tt class="docutils literal">with</tt> block is exited and,
in case a Python exception is raised, the error is pushed to the server, which
will cancel the COPY operation in progress.</p>
<div class="section" id="binary-format">
<h2>Binary format</h2>
<p>The <tt class="docutils literal">Copy</tt> object is also able to write data in <a class="reference external" href="https://www.postgresql.org/docs/13/sql-copy.html#id-1.9.3.55.9.4">binary format</a>: at Python
level this is entirely transparent:</p>
<pre class="code python literal-block">
<span class="k">with</span> <span class="n">cursor</span><span class="o">.</span><span class="n">copy</span><span class="p">(</span><span class="s2">&quot;COPY mytable FROM STDIN (FORMAT BINARY)&quot;</span><span class="p">)</span> <span class="k">as</span> <span class="n">copy</span><span class="p">:</span><span class="w">
</span> <span class="k">for</span> <span class="n">record</span> <span class="ow">in</span> <span class="n">generator</span><span class="p">:</span><span class="w">
</span> <span class="n">copy</span><span class="o">.</span><span class="n">write_row</span><span class="p">(</span><span class="n">record</span><span class="p">)</span>
</pre>
<p>which might be more efficient than the textual format, but requires more care
with the data types, as the server will not even perform an innocent
<tt class="docutils literal">int4</tt> -&gt; <tt class="docutils literal">int8</tt> cast for you.</p>
</div>
<div class="section" id="block-level-copy">
<h2>Block-level COPY</h2>
<p><tt class="docutils literal">psycopg2</tt> allows (only) to operate on a COPY stream using a Python
file-like objects: behind the scenes it reads one block of data from the
source and writes it to the destination:</p>
<pre class="code python literal-block">
<span class="c1"># From a file to the database</span><span class="w">
</span><span class="k">with</span> <span class="nb">open</span><span class="p">(</span><span class="s2">&quot;input.data&quot;</span><span class="p">)</span> <span class="k">as</span> <span class="n">f</span><span class="p">:</span><span class="w">
</span> <span class="n">cursor2</span><span class="o">.</span><span class="n">copy_expert</span><span class="p">(</span><span class="n">f</span><span class="p">,</span> <span class="s2">&quot;COPY mytable FROM STDIN&quot;</span><span class="p">)</span><span class="w">
</span><span class="c1"># From the database to a file</span><span class="w">
</span><span class="k">with</span> <span class="nb">open</span><span class="p">(</span><span class="s2">&quot;output.data&quot;</span><span class="p">,</span> <span class="s2">&quot;w&quot;</span><span class="p">)</span> <span class="k">as</span> <span class="n">f</span><span class="p">:</span><span class="w">
</span> <span class="n">cursor2</span><span class="o">.</span><span class="n">copy_expert</span><span class="p">(</span><span class="n">f</span><span class="p">,</span> <span class="s2">&quot;COPY mytable TO STDOUT&quot;</span><span class="p">)</span>
</pre>
<p>This way of operating is not lost, but now the responsibility of moving data
from one stream to the other is left to the user's code:</p>
<pre class="code python literal-block">
<span class="k">with</span> <span class="nb">open</span><span class="p">(</span><span class="s2">&quot;input.data&quot;</span><span class="p">)</span> <span class="k">as</span> <span class="n">f</span><span class="p">:</span><span class="w">
</span> <span class="k">with</span> <span class="n">cursor3</span><span class="o">.</span><span class="n">copy</span><span class="p">(</span><span class="s2">&quot;COPY mytable FROM STDIN&quot;</span><span class="p">)</span> <span class="k">as</span> <span class="n">copy</span><span class="p">:</span><span class="w">
</span> <span class="k">while</span> <span class="n">data</span> <span class="o">:=</span> <span class="n">f</span><span class="o">.</span><span class="n">read</span><span class="p">(</span><span class="n">SIZE</span><span class="p">):</span><span class="w">
</span> <span class="n">copy</span><span class="o">.</span><span class="n">write</span><span class="p">(</span><span class="n">data</span><span class="p">)</span><span class="w">
</span><span class="k">with</span> <span class="n">cursor3</span><span class="o">.</span><span class="n">copy</span><span class="p">(</span><span class="s2">&quot;COPY mytable TO STDOUT&quot;</span><span class="p">)</span> <span class="k">as</span> <span class="n">copy</span><span class="p">:</span><span class="w">
</span> <span class="k">with</span> <span class="nb">open</span><span class="p">(</span><span class="s2">&quot;output.data&quot;</span><span class="p">,</span> <span class="s2">&quot;wb&quot;</span><span class="p">)</span> <span class="k">as</span> <span class="n">f</span><span class="p">:</span><span class="w">
</span> <span class="k">while</span> <span class="n">data</span> <span class="o">:=</span> <span class="n">copy</span><span class="o">.</span><span class="n">read</span><span class="p">()</span><span class="w">
</span> <span class="n">f</span><span class="o">.</span><span class="n">write</span><span class="p">(</span><span class="n">data</span><span class="p">)</span>
</pre>
<p>While the new pattern is more verbose, it allows to produce and consume data
with interfaces different than the file one, whereas previously it would have
required to write some form of file-like adapter, blocking the copy in case no
data was ready. This inversion of control allows, finally, the use of...</p>
</div>
<div class="section" id="asynchronous-copy">
<h2>Asynchronous COPY</h2>
<p>If your data producer, either at rows level or at blocks level, is capable of
asynchronous operations, it is now possible to combine it asynchronously with
COPY using exactly the same pattern as the sync code, only sprinkling the
magic words here and there:</p>
<pre class="code python literal-block">
<span class="k">async</span> <span class="k">with</span> <span class="n">cursor</span><span class="o">.</span><span class="n">copy</span><span class="p">(</span><span class="s2">&quot;COPY mytable FROM STDIN (FORMAT BINARY)&quot;</span><span class="p">)</span> <span class="k">as</span> <span class="n">copy</span><span class="p">:</span><span class="w">
</span> <span class="k">async</span> <span class="k">for</span> <span class="n">record</span> <span class="ow">in</span> <span class="n">my_async_generator</span><span class="p">():</span><span class="w">
</span> <span class="k">await</span> <span class="n">copy</span><span class="o">.</span><span class="n">write_row</span><span class="p">(</span><span class="n">record</span><span class="p">)</span><span class="w">
</span><span class="k">async</span> <span class="k">with</span> <span class="n">async_producer</span><span class="p">()</span> <span class="k">as</span> <span class="n">f</span><span class="p">:</span><span class="w">
</span> <span class="k">async</span> <span class="k">with</span> <span class="n">cursor</span><span class="o">.</span><span class="n">copy</span><span class="p">(</span><span class="s2">&quot;COPY mytable FROM STDIN&quot;</span><span class="p">)</span> <span class="k">as</span> <span class="n">copy</span><span class="p">:</span><span class="w">
</span> <span class="k">while</span> <span class="n">data</span> <span class="o">:=</span> <span class="k">await</span> <span class="n">f</span><span class="o">.</span><span class="n">read</span><span class="p">()</span><span class="w">
</span> <span class="k">await</span> <span class="n">copy</span><span class="o">.</span><span class="n">write</span><span class="p">(</span><span class="n">data</span><span class="p">)</span>
</pre>
<p>which covers an important use case pretty much impossible to introduce in
<tt class="docutils literal">psycopg2</tt>.</p>
</div>
<div class="section" id="project-sponsorship">
<h2>Project sponsorship</h2>
<div class="sponsor docutils container">
<p>The new COPY support is one of several new features that are being
designed and implemented in <a class="reference external" href="/psycopg3/">psycopg3</a>. The project is currently <a class="reference external" href="https://github.com/psycopg/psycopg3">under
active development</a>: if there is enough support it will be possible to
work at the project full-time and bring it to release swiftly.</p>
<p>If you use Python and PostgreSQL, and you would like to support the
creation of the most advanced adapter between the two systems, <a class="reference external" href="https://github.com/sponsors/dvarrazzo/">please
consider becoming a sponsor</a>. Thank you!</p>
<iframe src="https://github.com/sponsors/dvarrazzo/button"
title="Sponsor the project" style="border: 0"
width="116" height="35">
</iframe></div>
</div>
</content></entry><entry><title>Psycopg 2.8.6 released</title><link href="https://www.psycopg.org/articles/2020/09/06/psycopg-286-released/" rel="alternate"></link><updated>2020-09-06T00:00:00Z</updated><author><name>Daniele Varrazzo</name></author><id>urn:uuid:95d68852-ec64-3745-82b4-defa6188cd5e</id><content type="html"><p>Psycopg 2.8.6 has been released.</p>
<!-- CUT-HERE -->
<p>You can download the files from the new release <a class="reference external" href="https://pypi.org/project/psycopg2/2.8.6/#files">from PyPI</a> or install it
with:</p>
<pre class="literal-block">
pip install --upgrade psycopg2
</pre>
<p>The changes included in the release are:</p>
<ul class="simple">
<li>Fixed memory leak changing connection encoding to the current one
(<a class="reference external" href="https://github.com/psycopg/psycopg2/issues/1101">ticket #1101</a>).</li>
<li>Fixed search of mxDateTime headers in virtualenvs (<a class="reference external" href="https://github.com/psycopg/psycopg2/issues/996">ticket #996</a>).</li>
<li>Added missing values from errorcodes (<a class="reference external" href="https://github.com/psycopg/psycopg2/issues/1133">ticket #1133</a>).</li>
<li><tt class="docutils literal">cursor.query</tt> reports the query of the last <tt class="docutils literal">COPY</tt> operation too
(<a class="reference external" href="https://github.com/psycopg/psycopg2/issues/1141">ticket #1141</a>).</li>
<li><tt class="docutils literal">errorcodes</tt> map and <tt class="docutils literal">errors</tt> classes updated to PostgreSQL 13.</li>
<li>Wheel package compiled against OpenSSL 1.1.1g.</li>
</ul>
<p>Thank you very much to everyone who gave their contribution!</p>
</content></entry><entry><title>Psycopg 2.8.5 released</title><link href="https://www.psycopg.org/articles/2020/04/06/psycopg-285-released/" rel="alternate"></link><updated>2020-04-06T00:00:00Z</updated><author><name>Daniele Varrazzo</name></author><id>urn:uuid:0d6cc67c-2767-3298-b069-34bc493f8e6b</id><content type="html"><p>Psycopg 2.8.5 has been released.</p>
<p>This release adds support for AIX and brings a few bug fixes.</p>
<!-- CUT-HERE -->
<p>You can download the files from the new release <a class="reference external" href="https://pypi.org/project/psycopg2/2.8.5/#files">from PyPI</a> or install it
with:</p>
<pre class="literal-block">
pip install --upgrade psycopg2
</pre>
<p>The changes included in the release are:</p>
<ul class="simple">
<li>Fixed use of <tt class="docutils literal">connection_factory</tt> and <tt class="docutils literal">cursor_factory</tt> together
(<a class="reference external" href="https://github.com/psycopg/psycopg2/issues/1019">ticket #1019</a>).</li>
<li>Added support for <tt class="docutils literal">logging.LoggerAdapter</tt> in
<tt class="docutils literal">LoggingConnection</tt> (<a class="reference external" href="https://github.com/psycopg/psycopg2/issues/1026">ticket #1026</a>).</li>
<li><tt class="docutils literal">Column</tt> objects in <tt class="docutils literal">cursor.description</tt> can be sliced (<a class="reference external" href="https://github.com/psycopg/psycopg2/issues/1034">ticket #1034</a>).</li>
<li>Added AIX support (<a class="reference external" href="https://github.com/psycopg/psycopg2/issues/1061">ticket #1061</a>).</li>
<li>Fixed <tt class="docutils literal">copy()</tt> of <tt class="docutils literal">DictCursor</tt> rows (<a class="reference external" href="https://github.com/psycopg/psycopg2/issues/1073">ticket #1073</a>).</li>
</ul>
<p>Happy hacking!</p>
</content></entry><entry><title>Psycopg 2.8.4 released</title><link href="https://www.psycopg.org/articles/2019/10/20/psycopg-284-released/" rel="alternate"></link><updated>2019-10-20T00:00:00Z</updated><author><name>Daniele Varrazzo</name></author><id>urn:uuid:f1c59860-afc2-3a75-bd1f-55eceed64247</id><content type="html"><p>Psycopg 2.8.4 has been released.</p>
<p>The release brings a few assorted bugfixes and adds support for Python 3.8 and PostgreSQL 12.</p>
<!-- CUT-HERE -->
<p>A more detailed changes list is</p>
<ul class="simple">
<li>Fixed building with Python 3.8 (<a class="reference external" href="https://github.com/psycopg/psycopg2/issues/854">ticket #854</a>).</li>
<li>Don't swallow keyboard interrupts on connect when a password is specified
in the connection string (<a class="reference external" href="https://github.com/psycopg/psycopg2/issues/898">ticket #898</a>).</li>
<li>Don't advance replication cursor when the message wasn't confirmed
(<a class="reference external" href="https://github.com/psycopg/psycopg2/issues/940">ticket #940</a>).</li>
<li>Fixed inclusion of <tt class="docutils literal">time.h</tt> on linux (<a class="reference external" href="https://github.com/psycopg/psycopg2/issues/951">ticket #951</a>).</li>
<li>Fixed int overflow for large values in <tt class="docutils literal">Column.table_oid</tt>
and <tt class="docutils literal">Column.type_code</tt> (<a class="reference external" href="https://github.com/psycopg/psycopg2/issues/961">ticket #961</a>).</li>
<li><tt class="docutils literal">errorcodes</tt> map and <tt class="docutils literal">errors</tt> classes updated to
PostgreSQL 12.</li>
<li>Wheel package compiled against OpenSSL 1.1.1d and PostgreSQL at least 11.4.</li>
</ul>
</content></entry><entry><title>Psycopg 2.8.3 released</title><link href="https://www.psycopg.org/articles/2019/06/14/psycopg-283-released/" rel="alternate"></link><updated>2019-06-14T00:00:00Z</updated><author><name>Daniele Varrazzo</name></author><id>urn:uuid:eaa9eab2-886c-35c0-8ce5-5f2d6cc2f665</id><content type="html"><p>We have released Psycopg 2.8.3, which includes a slight change to the logical replication.</p>
<p>Choosing the right frequency to send replication feedback messages from the client to the server was previously the developer's responsibility, with too many feedback messages being a waste of bandwidth and server resources, too few slowing down WAL cleanup and possibly preventing a server graceful shutdown.</p>
<!-- CUT-HERE -->
<p>Psycopg will now make sure that feedback is only sent after a certain period of time from the previous one, so that the client can simply call <tt class="docutils literal">send_feedback()</tt> at each message without the fear of overwhelming the server.</p>
<p>Further details are available in <a class="reference external" href="https://github.com/psycopg/psycopg2/pull/913">the MR</a> by <a class="reference external" href="https://github.com/CyberDem0n">Alexander Kukushkin</a> and <a class="reference external" href="https://github.com/a1exsh">Oleksandr Shulgin</a>. Thank you very much!</p>
<p>For completeness, the changes included in the release are:</p>
<ul class="simple">
<li>Added <em>interval_status</em> parameter to <tt class="docutils literal">start_replication()</tt> method and other facilities to send automatic replication keepalives at periodic intervals (<a class="reference external" href="https://github.com/psycopg/psycopg2/issues/913">ticket #913</a>).</li>
<li>Fixed namedtuples caching introduced in 2.8 (<a class="reference external" href="https://github.com/psycopg/psycopg2/issues/928">ticket #928</a>).</li>
</ul>
</content></entry><entry><title>Psycopg 2.8.1, 2.8.2 released</title><link href="https://www.psycopg.org/articles/2019/04/14/psycopg-281-282-released/" rel="alternate"></link><updated>2019-04-14T00:00:00Z</updated><author><name>Daniele Varrazzo</name></author><id>urn:uuid:35398f6f-d9ff-3e0c-bbfd-1c0172c42437</id><content type="html"><p>Hello,</p>
<p>We have just released Psycopg 2.8.2; a few days ago Psycopg 2.8.1 was released.</p>
<p>Some of the bugs addressed are ordinary teething problem with the 2.8
release, but an important change landed with 2.8.2: binary packages
now ship with OpenSSL 1.1 instead of 1.0. This should fix concurrency
problems on connection experienced both on Windows and Linux. Many
thanks to Matthew Brett and Jason Erickson for this improvement!</p>
<!-- CUT-HERE -->
<p>The combined list of changes is:</p>
<ul class="simple">
<li>Fixed &quot;there's no async cursor&quot; error polling a connection with no cursor
(<a class="reference external" href="https://github.com/psycopg/psycopg2/issues/887">ticket #887</a>).</li>
<li>Fixed <tt class="docutils literal">RealDictCursor</tt> when there are repeated columns (<a class="reference external" href="https://github.com/psycopg/psycopg2/issues/884">ticket #884</a>).</li>
<li>Fixed <tt class="docutils literal">RealDictRow</tt> modifiability (<a class="reference external" href="https://github.com/psycopg/psycopg2/issues/886">ticket #886</a>).</li>
<li>Binary packages built with OpenSSL 1.1.1b. Should fix concurrency problems
(<a class="reference external" href="https://github.com/psycopg/psycopg2/issues/543">ticket #543</a>, <a class="reference external" href="https://github.com/psycopg/psycopg2/issues/836">ticket #836</a>).</li>
</ul>
<p>Happy hacking!</p>
</content></entry><entry><title>Psycopg 2.8 released</title><link href="https://www.psycopg.org/articles/2019/04/04/psycopg-28-released/" rel="alternate"></link><updated>2019-04-04T00:00:00Z</updated><author><name>Daniele Varrazzo</name></author><id>urn:uuid:51a61d12-fff7-38d4-9d87-3210fac55ec9</id><content type="html"><p>After about two years from the previous major release, psycopg 2.8 is finally here!</p>
<p>Among the highlights, <a class="reference external" href="/docs/errors.html">PostgreSQL errors are now mapped to Python exceptions</a> for a more idiomatic way to handle them. Several additions allow a better insight of the <a class="reference external" href="/docs/extensions.html#psycopg2.extensions.ConnectionInfo">connection status</a> and <a class="reference external" href="/docs/extensions.html#psycopg2.extensions.Column">query results</a>.</p>
<p>Behind the scene, asynchronous communication and concurrency received several improvements, and dropping support for older versions of Python gave the chance to refactor and modernise the codebase (with the especial help from Jon Dufresne who ruthlessly butchered our code into a streamlined pulp).</p>
<p>Thank you very much to everyone contributing so far. Happy hacking!</p>
<!-- CUT-HERE -->
<p>You can download as usual from the canonical urls:</p>
<ul class="simple">
<li><a class="reference external" href="https://pypi.org/packages/source/p/psycopg2/psycopg2-2.8.tar.gz">source package</a></li>
<li><a class="reference external" href="https://pypi.org/packages/source/p/psycopg2/psycopg2-2.8.tar.gz.asc">signature</a></li>
</ul>
<p>Or just use <tt class="docutils literal">pip</tt>:</p>
<pre class="literal-block">
pip install psycopg2
</pre>
<p>New features:</p>
<ul class="simple">
<li>Added <tt class="docutils literal">errors</tt> module. Every PostgreSQL error is converted into a specific exception class (<a class="reference external" href="https://github.com/psycopg/psycopg2/issues/682">ticket #682</a>).</li>
<li>Added <tt class="docutils literal">encrypt_password()</tt> function (<a class="reference external" href="https://github.com/psycopg/psycopg2/issues/576">ticket #576</a>).</li>
<li>Added <tt class="docutils literal">BYTES</tt> adapter to manage databases with mixed encodings on Python 3 (<a class="reference external" href="https://github.com/psycopg/psycopg2/issues/835">ticket #835</a>).</li>
<li>Added <tt class="docutils literal">table_oid</tt> and <tt class="docutils literal">table_column</tt> attributes on <tt class="docutils literal">cursor.description</tt> items (<a class="reference external" href="https://github.com/psycopg/psycopg2/issues/661">ticket #661</a>).</li>
<li>Added <tt class="docutils literal">connection.info</tt> object to retrieve various PostgreSQL connection information (<a class="reference external" href="https://github.com/psycopg/psycopg2/issues/726">ticket #726</a>).</li>
<li>Added <tt class="docutils literal">get_native_connection()</tt> to expose the raw <tt class="docutils literal">PGconn</tt> structure to C extensions via Capsule (<a class="reference external" href="https://github.com/psycopg/psycopg2/issues/782">ticket #782</a>).</li>
<li>Added <tt class="docutils literal">pgconn_ptr</tt> and <tt class="docutils literal">pgresult_ptr</tt> to expose raw C structures to Python and interact with libpq via ctypes (<a class="reference external" href="https://github.com/psycopg/psycopg2/issues/782">ticket #782</a>).</li>
<li><tt class="docutils literal">sql.Identifier</tt> can represent qualified names in SQL composition (<a class="reference external" href="https://github.com/psycopg/psycopg2/issues/732">ticket #732</a>).</li>
<li>Added <tt class="docutils literal">ReplicationCursor.wal_end</tt> attribute (<a class="reference external" href="https://github.com/psycopg/psycopg2/issues/800">ticket #800</a>).</li>
<li>Added <em>fetch</em> parameter to <tt class="docutils literal">execute_values()</tt> function (<a class="reference external" href="https://github.com/psycopg/psycopg2/issues/813">ticket #813</a>).</li>
<li><tt class="docutils literal">str()</tt> on <tt class="docutils literal">Range</tt> produces a human-readable representation (<a class="reference external" href="https://github.com/psycopg/psycopg2/issues/773">ticket #773</a>).</li>
<li><tt class="docutils literal">DictCursor</tt> and <tt class="docutils literal">RealDictCursor</tt> rows maintain columns order (<a class="reference external" href="https://github.com/psycopg/psycopg2/issues/177">ticket #177</a>).</li>
<li>Added <tt class="docutils literal">severity_nonlocalized</tt> attribute on the <tt class="docutils literal">Diagnostics</tt> object (<a class="reference external" href="https://github.com/psycopg/psycopg2/issues/783">ticket #783</a>).</li>
<li>More efficient <tt class="docutils literal">NamedTupleCursor</tt> (<a class="reference external" href="https://github.com/psycopg/psycopg2/issues/838">ticket #838</a>).</li>
</ul>
<p>Bug fixes:</p>
<ul class="simple">
<li>Fixed connections occasionally broken by the unrelated use of the multiprocessing module (<a class="reference external" href="https://github.com/psycopg/psycopg2/issues/829">ticket #829</a>).</li>
<li>Fixed async communication blocking if results are returned in different chunks, e.g. with notices interspersed to the results (<a class="reference external" href="https://github.com/psycopg/psycopg2/issues/856">ticket #856</a>).</li>
<li>Fixed adaptation of numeric subclasses such as <tt class="docutils literal">IntEnum</tt> (<a class="reference external" href="https://github.com/psycopg/psycopg2/issues/591">ticket #591</a>).</li>
</ul>
<p>Other changes:</p>
<ul class="simple">
<li>Dropped support for Python 2.6, 3.2, 3.3.</li>
<li>Dropped <tt class="docutils literal">psycopg1</tt> module.</li>
<li>Dropped deprecated <tt class="docutils literal">register_tstz_w_secs()</tt> (was previously a no-op).</li>
<li>Dropped deprecated <tt class="docutils literal">PersistentConnectionPool</tt>. This pool class was mostly
designed to interact with Zope. Use <tt class="docutils literal">ZPsycopgDA.pool</tt> instead.</li>
<li>Binary packages no longer installed by default. The <tt class="docutils literal"><span class="pre">psycopg2-binary</span></tt> package must be used explicitly.</li>
<li>Dropped <tt class="docutils literal">PSYCOPG_DISPLAY_SIZE</tt> build parameter.</li>
<li>Dropped support for mxDateTime as the default date and time adapter. mxDatetime support continues to be available as an alternative to Python's builtin datetime.</li>
<li>No longer use 2to3 during installation for Python 2 &amp; 3 compatibility. All source files are now compatible with Python 2 &amp; 3 as is.</li>
<li>The <tt class="docutils literal">psycopg2.test</tt> package is no longer installed by <tt class="docutils literal">python setup.py install</tt>.</li>
<li>Wheel package compiled against OpenSSL 1.0.2r and PostgreSQL 11.2 libpq.</li>
</ul>
</content></entry><entry><title>Psycopg 2.7.5 released</title><link href="https://www.psycopg.org/articles/2018/06/17/psycopg-275-released/" rel="alternate"></link><updated>2018-06-17T00:00:00Z</updated><author><name>Daniele Varrazzo</name></author><id>urn:uuid:1f5d57e4-8e09-3f5d-86b0-bbb69911d846</id><content type="html"><p>psycopg2 version 2.7.5 has been released, fixing a few bugs found in the last months:</p>
<!-- CUT-HERE -->
<p>Summary of changes:</p>
<ul class="simple">
<li>Allow non-ascii chars in namedtuple fields (regression introduced fixing <a class="reference external" href="https://github.com/psycopg/psycopg2/issues/211">ticket #211</a>).</li>
<li>Fixed adaptation of arrays of arrays of nulls (<a class="reference external" href="https://github.com/psycopg/psycopg2/issues/325">ticket #325</a>).</li>
<li>Fixed building on Solaris 11 and derivatives such as SmartOS and illumos (<a class="reference external" href="https://github.com/psycopg/psycopg2/issues/677">ticket #677</a>).</li>
<li>Maybe fixed building on MSYS2 (as reported in <a class="reference external" href="https://github.com/psycopg/psycopg2/issues/658">ticket #658</a>).</li>
<li>Allow string subclasses in connection and other places (<a class="reference external" href="https://github.com/psycopg/psycopg2/issues/679">ticket #679</a>).</li>
<li>Don't raise an exception closing an unused named cursor (<a class="reference external" href="https://github.com/psycopg/psycopg2/issues/716">ticket #716</a>).</li>
<li>Wheel package compiled against PostgreSQL 10.4 libpq and OpenSSL 1.0.2o.</li>
</ul>
<p>You can install psycopg2 <a class="reference external" href="https://pypi.python.org/pypi/psycopg2">from PyPI</a> or grab the new code from:</p>
<ul class="simple">
<li><a class="reference external" href="https://pypi.org/packages/source/p/psycopg2/psycopg2-2.7.5.tar.gz">source package</a></li>
<li><a class="reference external" href="https://pypi.org/packages/source/p/psycopg2/psycopg2-2.7.5.tar.gz.asc">signature</a></li>
</ul>
<p>Happy hacking!</p>
</content></entry><entry><title>Psycopg 2.7.4 released</title><link href="https://www.psycopg.org/articles/2018/02/08/psycopg-274-released/" rel="alternate"></link><updated>2018-02-08T00:00:00Z</updated><author><name>Daniele Varrazzo</name></author><id>urn:uuid:60fc514c-be7a-3b25-9fda-2eeabb8d38e9</id><content type="html"><p>We have released Psycopg version 2.7.4, bringing a few bug fixes... and working out the problem with Wheel packages.</p>
<!-- CUT-HERE -->
<div class="section" id="what-s-the-problem-with-wheels">
<h2>What's the problem with Wheels?</h2>
<p><a class="reference external" href="https://pythonwheels.com/">Wheel packages</a> are a Python standard to distribute self-contained binary package. They are especially convenient for packages containing C extensions, such as Psycopg, and for packages depending on external C libraries... such as Psycopg, because the package will contain a binary, pre-compiled, version of the C extension and all the depending libraries (excluding a list of libraries expected to be found on any system and with a versioned ABI, such as libc). Since the release of the wheel packages with <a class="reference external" href="/articles/2017/03/01/psycopg-27-released/">psycopg 2.7</a> it has been possible to install Psycopg without the <a class="reference external" href="/docs/install.html#build-prerequisites">prerequisites</a> of a C compiler and of Python/Postgres dev packages at build time, and the need of a system libpq at runtime.</p>
<p>Unfortunately, after the packages were released, <a class="reference external" href="https://github.com/psycopg/psycopg2/issues/543">it was reported</a> of occasional segfaults of Python processes using Psycopg from the Wheel package. This was traced to the use of the Python <tt class="docutils literal">ssl</tt> library concurrently with Psycopg opening connections, for instance in a multithread program opening concurrently <tt class="docutils literal">https</tt> resources and database connections. The problem seems caused by a non-reentrant OpenSSL initialization function (unfortunately invoked by libpq at every connection) and the fact that the Python process ends up binding two different versions of the <tt class="docutils literal">libssl</tt> library: the system one via the Python <tt class="docutils literal">ssl</tt> library (e.g. imported by <tt class="docutils literal">urllib</tt> or <tt class="docutils literal">requests</tt>) and the Wheel one imported by the <tt class="docutils literal">libpq</tt>, in turn imported by <tt class="docutils literal">psycopg2</tt>.</p>
<p>While the problem doesn't affect many users, a library behaving unreliably in combination with part of the stdlib is a situation less than optimal. The workaround is to force installing Psycopg from source, but this must be specified explicitly in the project dependencies (e.g. using the <tt class="docutils literal"><span class="pre">--no-binary</span></tt> flag in the <tt class="docutils literal">pip</tt> command line or in the <tt class="docutils literal">requirements.txt</tt> file); the Python packaging system doesn't really have a way to declare something like &quot;install a package preferably from source&quot;... so we had to create one ourselves.</p>
<p>Starting with Psycopg 2.7.4, we are releasing two different packages on PyPI: <a class="reference external" href="https://pypi.python.org/pypi/psycopg2/">psycopg2</a> and <a class="reference external" href="https://pypi.python.org/pypi/psycopg2-binary/">psycopg2-binary</a>. The latter is a Wheels-only package, with a behaviour identical to the classic one – the different name is used only in installation (it is installed by <tt class="docutils literal">pip install <span class="pre">psycopg2-binary</span></tt>, but still imported with <tt class="docutils literal">import psycopg2</tt> in Python).</p>
<p>For the lifespan of the Psycopg 2.7 cycle, the <tt class="docutils literal">psycopg2</tt> PyPI package will still contain wheel packages, but starting from Psycopg 2.8 it will become again a source-only package. Starting from Psycopg 2.7.4, if the package is installed as binary from the <tt class="docutils literal">psycopg2</tt> PyPI entry, a warning will be reported on import:</p>
<blockquote>
The psycopg2 wheel package will be renamed from release 2.8; in order to keep installing from binary please use &quot;pip install psycopg2-binary&quot; instead. For details see: &lt;/docs/install.html#binary-install-from-pypi&gt;.</blockquote>
<p>The choices for the users are then two:</p>
<ul class="simple">
<li>if the program works fine with the Wheel packages, and the convenience of the binary distribution is preferred, it is possible to <a class="reference external" href="/docs/install.html#binary-install-from-pypi">specify the dependency on the binary package</a> using the <tt class="docutils literal"><span class="pre">psycopg2-binary</span></tt> instead of the <tt class="docutils literal">psycopg2</tt> PyPI package. No change to the program code is needed;</li>
<li>if there are concerns about the unreliability of the Wheels package, it is advised to <a class="reference external" href="/docs/install.html#disabling-wheel-packages-for-psycopg-2-7">force installation from source</a>. This requires the presence of build tools and runtime libraries on the client, but again it requires no change to the code.</li>
</ul>
<p>We hope this solution will suggest the default use of a reliable version of the library, while still allowing the convenience of a dependencies-free package. Feedback is welcome on the <a class="reference external" href="https://www.postgresql.org/list/psycopg/">mailing list</a>.</p>
</div>
<div class="section" id="other-changes-in-this-version">
<h2>Other changes in this version</h2>
<ul class="simple">
<li>Convert fields names into valid Python identifiers in
<tt class="docutils literal">NamedTupleCursor</tt> (<a class="reference external" href="https://github.com/psycopg/psycopg2/issues/211">ticket #211</a>).</li>
<li>Fixed Solaris 10 support (<a class="reference external" href="https://github.com/psycopg/psycopg2/issues/532">ticket #532</a>).</li>
<li><tt class="docutils literal">cursor.mogrify()</tt> can be called on closed cursors (<a class="reference external" href="https://github.com/psycopg/psycopg2/issues/579">ticket #579</a>).</li>
<li>Fixed setting session characteristics in corner cases on autocommit
connections (<a class="reference external" href="https://github.com/psycopg/psycopg2/issues/580">ticket #580</a>).</li>
<li>Fixed <tt class="docutils literal">MinTimeLoggingCursor</tt> on Python 3 (<a class="reference external" href="https://github.com/psycopg/psycopg2/issues/609">ticket #609</a>).</li>
<li>Fixed parsing of array of points as floats (<a class="reference external" href="https://github.com/psycopg/psycopg2/issues/613">ticket #613</a>).</li>
<li>Fixed <tt class="docutils literal">__libpq_version__</tt> building with libpq &gt;= 10.1
(<a class="reference external" href="https://github.com/psycopg/psycopg2/issues/632">ticket 632</a>).</li>
<li>Fixed <tt class="docutils literal">rowcount</tt> after <tt class="docutils literal">executemany()</tt> with <tt class="docutils literal">RETURNING</tt>
statements (<a class="reference external" href="https://github.com/psycopg/psycopg2/issues/633">ticket 633</a>).</li>
<li>Fixed compatibility problem with pypy3 (<a class="reference external" href="https://github.com/psycopg/psycopg2/issues/649">ticket #649</a>).</li>
<li>Wheel packages compiled against PostgreSQL 10.1 libpq and OpenSSL 1.0.2n.</li>
<li>Wheel packages for Python 2.6 no more available (support dropped from
wheel building infrastructure).</li>
</ul>
</div>
</content></entry><entry><title>Psycopg 2.7.3.2 released</title><link href="https://www.psycopg.org/articles/2017/10/24/psycopg-2732-released/" rel="alternate"></link><updated>2017-10-24T00:00:00Z</updated><author><name>Daniele Varrazzo</name></author><id>urn:uuid:d04b31ed-bb14-30e7-8adb-2658369c7f3a</id><content type="html"><p>Following the release of <a class="reference external" href="https://www.postgresql.org/about/news/1786/">PostgreSQL 10</a> we have released new binary packages as Psycopg 2.7.3.2. There are no code changes from Psycopg 2.7.3, but the new binary packages ship with the PostgreSQL 10 client library, enabling the use of new features such as <a class="reference external" href="http://paquier.xyz/postgresql-2/postgres-10-multi-host-connstr/">multiple hosts in connection string</a>, <a class="reference external" href="http://paquier.xyz/postgresql-2/postgres-10-libpq-read-write/">read-only mode</a>, <a class="reference external" href="https://www.postgresql.org/docs/10/static/sasl-authentication.html#sasl-scram-sha256">SCRAM-SHA-256 authentication</a>.</p>
<!-- CUT-HERE -->
<p>The packages are already <a class="reference external" href="https://pypi.python.org/pypi/psycopg2/2.7.3.2">available on PyPI</a> and automatically installed by <tt class="docutils literal">pip install psycopg2</tt> (if your pip is up-to-date). You can grab the source code from:</p>
<ul class="simple">
<li><a class="reference external" href="https://pypi.org/packages/source/p/psycopg2/psycopg2-2.7.3.2.tar.gz">source package</a></li>
<li><a class="reference external" href="https://pypi.org/packages/source/p/psycopg2/psycopg2-2.7.3.2.tar.gz.asc">signature</a></li>
</ul>
<p>Happy hacking!</p>
</content></entry><entry><title>Psycopg 2.7.3.1 released</title><link href="https://www.psycopg.org/articles/2017/08/26/psycopg-2731-released/" rel="alternate"></link><updated>2017-08-26T00:00:00Z</updated><author><name>Daniele Varrazzo</name></author><id>urn:uuid:5101b867-469b-3bd0-81ff-58baeeddc374</id><content type="html"><p>We have released psycopg2 release 2.7.3.1 as a new build of psycopg 2.7.3. The new build only affects the wheel packages and contains no change in the code.</p>
<p>The release 2.7.3.1 fixes <a class="reference external" href="https://github.com/psycopg/psycopg2-wheels/issues/2">psycopg2-wheels bug #2</a> which was in turn caused by <a class="reference external" href="https://github.com/pypa/auditwheel/issues/80">auditwheel bug #80</a>, resulting in incompatibility with glibc 2.26. The problem only affects Linux wheels users, it doesn't affect Windows, OSX, or user installing psycopg2 from source.</p>
<!-- CUT-HERE -->
<p>You can install psycopg2 <a class="reference external" href="https://pypi.python.org/pypi/psycopg2">from PyPI</a> or grab the new code from:</p>
<ul class="simple">
<li><a class="reference external" href="https://pypi.org/packages/source/p/psycopg2/psycopg2-2.7.3.1.tar.gz">source package</a></li>
<li><a class="reference external" href="https://pypi.org/packages/source/p/psycopg2/psycopg2-2.7.3.1.tar.gz.asc">signature</a></li>
</ul>
</content></entry><entry><title>Psycopg 2.7.3 released</title><link href="https://www.psycopg.org/articles/2017/07/24/psycopg-273-released/" rel="alternate"></link><updated>2017-07-24T00:00:00Z</updated><author><name>Daniele Varrazzo</name></author><id>urn:uuid:f522218f-040d-337a-bdb8-df17ba00dc99</id><content type="html"><p>A quick release to fix a regression found in <a class="reference external" href="/psycopg/articles/2017/07/22/psycopg-272-released/">psycopg 2.7.2</a>:</p>
<ul class="simple">
<li>Restored default <tt class="docutils literal">timestamptz[]</tt> typecasting to Python <tt class="docutils literal">datetime</tt>.
Regression introduced in Psycopg 2.7.2 (<a class="reference external" href="https://github.com/psycopg/psycopg2/issues/578">ticket #578</a>).</li>
</ul>
<!-- CUT-HERE -->
<p>You can install psycopg2 <a class="reference external" href="https://pypi.python.org/pypi/psycopg2">from PyPI</a> or grab the new code from:</p>
<ul class="simple">
<li><a class="reference external" href="https://pypi.org/packages/source/p/psycopg2/psycopg2-2.7.3.tar.gz">source package</a></li>
<li><a class="reference external" href="https://pypi.org/packages/source/p/psycopg2/psycopg2-2.7.3.tar.gz.asc">signature</a></li>
</ul>
<p>Happy hacking!</p>
</content></entry><entry><title>Psycopg 2.7.2 released</title><link href="https://www.psycopg.org/articles/2017/07/22/psycopg-272-released/" rel="alternate"></link><updated>2017-07-22T00:00:00Z</updated><author><name>Daniele Varrazzo</name></author><id>urn:uuid:5ad9993a-c322-3804-8675-286812d7f5c9</id><content type="html"><p>Releasing psycopg2 version 2.7.2: a release fixing a host of bugs found in the last 3-4 months.</p>
<!-- CUT-HERE -->
<p>Summary of changes:</p>
<ul class="simple">
<li>Fixed inconsistent state in externally closed connections (<a class="reference external" href="https://github.com/psycopg/psycopg2/issues/263">ticket #263</a>, <a class="reference external" href="https://github.com/psycopg/psycopg2/issues/311">ticket #311</a>, <a class="reference external" href="https://github.com/psycopg/psycopg2/issues/443">ticket #443</a>). Was fixed in 2.6.2 but not included in 2.7 by mistake.</li>
<li>Fixed Python exceptions propagation in green callback (<a class="reference external" href="https://github.com/psycopg/psycopg2/issues/410">ticket #410</a>).</li>
<li>Don't display the password in <tt class="docutils literal">connection.dsn</tt> when the connection string is specified as an URI (<a class="reference external" href="https://github.com/psycopg/psycopg2/issues/528">ticket #528</a>).</li>
<li>Return objects with timezone parsing &quot;infinity&quot; <tt class="docutils literal">timestamptz</tt> (<a class="reference external" href="https://github.com/psycopg/psycopg2/issues/536">ticket #536</a>).</li>
<li>Dropped dependency on VC9 runtime on Windows binary packages (<a class="reference external" href="https://github.com/psycopg/psycopg2/issues/541">ticket #541</a>).</li>
<li>Fixed segfault in <tt class="docutils literal">lobject()</tt> when <em>mode</em>=<tt class="docutils literal">None</tt> (<a class="reference external" href="https://github.com/psycopg/psycopg2/issues/544">ticket #544</a>).</li>
<li>Fixed <tt class="docutils literal">lobject()</tt> keyword argument <em>lobject_factory</em> (<a class="reference external" href="https://github.com/psycopg/psycopg2/issues/545">ticket #545</a>).</li>
<li>Fixed <tt class="docutils literal">ReplicationCursor.consume_stream()</tt> <em>keepalive_interval</em> argument (<a class="reference external" href="https://github.com/psycopg/psycopg2/issues/547">ticket #547</a>).</li>
<li>Maybe fixed random import error on Python 3.6 in multiprocess environment (<a class="reference external" href="https://github.com/psycopg/psycopg2/issues/550">ticket #550</a>).</li>
<li>Fixed random <tt class="docutils literal">SystemError</tt> upon receiving abort signal (<a class="reference external" href="https://github.com/psycopg/psycopg2/issues/551">ticket #551</a>).</li>
<li>Accept <tt class="docutils literal">sql.Composable</tt> objects in <tt class="docutils literal">ReplicationCursor.start_replication_expert()</tt> (<a class="reference external" href="https://github.com/psycopg/psycopg2/issues/554">ticket 554</a>).</li>
<li>Parse intervals returned as microseconds from Redshift (<a class="reference external" href="https://github.com/psycopg/psycopg2/issues/558">ticket #558</a>).</li>
<li>Added <tt class="docutils literal">Json.prepare()</tt> method to consider connection params when adapting (<a class="reference external" href="https://github.com/psycopg/psycopg2/issues/562">ticket #562</a>).</li>
<li><tt class="docutils literal">errorcodes</tt> map updated to PostgreSQL 10 beta 1.</li>
</ul>
<p>You can install psycopg2 <a class="reference external" href="https://pypi.python.org/pypi/psycopg2">from PyPI</a> or grab the new code from:</p>
<ul class="simple">
<li><a class="reference external" href="https://pypi.org/packages/source/p/psycopg2/psycopg2-2.7.2.tar.gz">source package</a></li>
<li><a class="reference external" href="https://pypi.org/packages/source/p/psycopg2/psycopg2-2.7.2.tar.gz.asc">signature</a></li>
</ul>
<p>Happy hacking!</p>
</content></entry><entry><title>Psycopg 2.7.1 released</title><link href="https://www.psycopg.org/articles/2017/03/13/psycopg-271-released/" rel="alternate"></link><updated>2017-03-13T00:00:00Z</updated><author><name>Daniele Varrazzo</name></author><id>urn:uuid:385e4a6f-e45c-37ac-9180-c30115a6a77b</id><content type="html"><p>A quick bugfix release to solve some teething problems with some of the changes introduced in 2.7:</p>
<ul class="simple">
<li>Ignore None arguments passed to <tt class="docutils literal">connect()</tt> and <tt class="docutils literal">make_dsn()</tt> (<a class="reference external" href="https://github.com/psycopg/psycopg2/issues/517">ticket #517</a>).</li>
<li>OpenSSL upgraded from major version 0.9.8 to 1.0.2 in the Linux wheel packages (<a class="reference external" href="https://github.com/psycopg/psycopg2/issues/518">ticket #518</a>).</li>
<li>Fixed build with libpq versions &lt; 9.3 (<a class="reference external" href="https://github.com/psycopg/psycopg2/issues/520">ticket #520</a>).</li>
</ul>
<!-- CUT-HERE -->
<p>Code fresh to grab at:</p>
<ul class="simple">
<li><a class="reference external" href="https://pypi.org/packages/source/p/psycopg2/psycopg2-2.7.1.tar.gz">source package</a></li>
<li><a class="reference external" href="https://pypi.org/packages/source/p/psycopg2/psycopg2-2.7.1.tar.gz.asc">signature</a></li>
</ul>
<p>Enjoy the release!</p>
</content></entry><entry><title>Psycopg 2.7 released</title><link href="https://www.psycopg.org/articles/2017/03/01/psycopg-27-released/" rel="alternate"></link><updated>2017-03-01T00:00:00Z</updated><author><name>Daniele Varrazzo</name></author><id>urn:uuid:851acb9c-c295-3fbd-81d5-8bbb4765e75a</id><content type="html"><p>Finally here! Thank you very much for waiting so long: we have finally released Psycopg 2.7!</p>
<p>Buzzwords:</p>
<ul class="simple">
<li><em>Faster!</em> Helps generating SQL for repeatedly executed statements and faster Unicode decoding.</li>
<li><em>Safer!</em> Helps generating dynamic SQL statements with variable table and field names.</li>
<li><em>Easier!</em> Use the binary package to avoid the need of C compiler, pg_config, libpq required on the clients.</li>
<li><em>Replication!</em> Support for PostgreSQL physical and logical replication.</li>
<li><em>Plays-better-with-pgbouncer-at-transaction-pooling-level!</em> This.</li>
</ul>
<!-- CUT-HERE -->
<p>You can download as usual from the canonical urls:</p>
<ul class="simple">
<li><a class="reference external" href="https://pypi.org/packages/source/p/psycopg2/psycopg2-2.7.tar.gz">source package</a></li>
<li><a class="reference external" href="https://pypi.org/packages/source/p/psycopg2/psycopg2-2.7.tar.gz.asc">signature</a></li>
</ul>
<p>Or just use <tt class="docutils literal">pip</tt>:</p>
<pre class="literal-block">