DurationField output format #8532

sevdog · 2022-06-24T09:14:12Z

Description

As explained in #8527 this PR will provide the ability to choose the output format for DurationFields.

Since Python builtim timedelta object does not have any format function we are going to allow only formats which are already defined in Django codebase in the package django.utils.duration:

duration_string
duration_iso_string

tomchristie · 2022-06-24T10:03:37Z

docs/api-guide/fields.md

 - `max_value` Validate that the duration provided is no greater than this value.
 - `min_value` Validate that the duration provided is no less than this value.

+#### `DurationField` format strings
+Format strings may either be the special string `'iso-8601'`, which indicates that [ISO 8601][iso8601] style intervals should be used (eg `'P4DT1H15M20S'`), or the special string `'standard'`, which indicates that Django interval format `'[DD] [HH:[MM:]]ss[.uuuuuu]'` hould be used (eg: `'4 1:15:20'`).


Suggested change

Format strings may either be the special string `'iso-8601'`, which indicates that [ISO 8601][iso8601] style intervals should be used (eg `'P4DT1H15M20S'`), or the special string `'standard'`, which indicates that Django interval format `'[DD] [HH:[MM:]]ss[.uuuuuu]'` hould be used (eg: `'4 1:15:20'`).

Format strings may either be the special string `'iso-8601'`, which indicates that [ISO 8601][iso8601] style intervals should be used (eg `'P4DT1H15M20S'`), or the special string `'standard'`, which indicates that Django interval format `'[DD] [HH:[MM:]]ss[.uuuuuu]'` should be used (eg: `'4 1:15:20'`).

Sorry for the typo 😅

I have fixed it locally, will update as soon as I have other elements to push because I would like to keep the history clear without having to perform too many rebase.

Okay. (Incidentally, all pull requests are squashed, so that doesn't matter at all from my perspective.)

tomchristie · 2022-06-24T10:05:20Z

docs/api-guide/settings.md

+
+#### DURATION_FORMAT
+
+A format string that should be used by default for rendering the output of `DurationField` serializer fields.  If `None`, then `DurationField` serializer fields will return Python `timedelta` objects, and the duration encoding will be determined by the renderer.


If None, then DurationField serializer fields will return Python timedelta objects, and the duration encoding will be determined by the renderer.

What output style does this produce with the current JSONRenderer?

Using the current JSONRenderer, which uses JSONEncoder it will output the string value of "total seconds" (doc).

django-rest-framework/rest_framework/utils/encoders.py

Lines 39 to 40 in 129890a

elif isinstance(obj, datetime.timedelta):

return str(obj.total_seconds())

Using the same example value will be:

from datetime import timedelta d = timedelta(days=4, hours=1, minutes=15, seconds=20) print(str(d.total_seconds()) # 350120.0

tomchristie · 2022-06-24T10:05:44Z

Thanks - good to talk this through from a docs-first perspective.

stale · 2022-10-16T13:40:55Z

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

pietzschke · 2024-06-03T21:20:25Z

Can we reopen this?

pietzschke · 2024-06-05T09:23:39Z

I think it's not possible to contribute to the pr to solve the conflicts, @sevdog would you be able to do it?
Otherwise i will create a new one and do the fixes.

sevdog · 2024-06-05T09:33:05Z

Just rebased the PR, the "conflict" was a very silly issue in the doc file 🤷‍♂️

CI is failing due to a change in django-main branch, it should be addressed in an other PR.

auvipy

I saw FAILED tests/test_fields.py::TestISOOutputFormatDurationField::test_outputs
=========== 1 failed,

sevdog · 2024-07-09T16:42:34Z

Ok, fixed the test. It was a long lasting double typo (wrong class and wrong format).

docs/api-guide/fields.md

auvipy · 2024-07-11T17:37:37Z

we should merge this only in a major release

stale · 2025-04-26T23:30:18Z

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

ulgens · 2025-04-27T06:40:17Z

The fix seems still relevant.

auvipy

I'm not sure if this will be accepted or not, but if it is accepted, fixing the merge conflict should be enough

sevdog · 2025-04-28T10:10:38Z

Conflict revolved.

peterthomassen

Thanks for resolving the conflict; some more comments here!

peterthomassen · 2025-04-28T10:34:19Z

docs/api-guide/fields.md

+* `format` - A string representing the output format.  If not specified, this defaults to the same value as the `DURATION_FORMAT` settings key, which will be `'standard'` unless set. Setting to a format string indicates that `to_representation` return values should be coerced to string output. Format strings are described below. Setting this value to `None` indicates that Python `timedelta` objects should be returned by `to_representation`. In this case the date encoding will be determined by the renderer.
 * `max_value` Validate that the duration provided is no greater than this value.
 * `min_value` Validate that the duration provided is no less than this value.

+#### `DurationField` format strings
+Format strings may either be the special string `'iso-8601'`, which indicates that [ISO 8601][iso8601] style intervals should be used (eg `'P4DT1H15M20S'`), or the special string `'standard'`, which indicates that Django interval format `'[DD] [HH:[MM:]]ss[.uuuuuu]'` should be used (eg: `'4 1:15:20'`).


I find the use of the term "format string" very confusing here. The term means something else in Python, referring to string with placeholder variables that get filled in. That sounds like one could set format="%h:%m:%s" to get the duration in hours, minutes, seconds, but that is not the case. Please find another word, like "format family", "format variety", ... or something.

Good point, how about format_name?

sounds good!

I used the "format" name just to keep uniformity with date/time fields, because to me it looked more elegant.

Since there is nothing like strftime/strptime for timedelta this may also just be a flag if it is clear/readable enough.

peterthomassen · 2025-04-28T10:35:39Z

rest_framework/__init__.py

@@ -21,6 +21,7 @@

 # Default datetime input and output formats
 ISO_8601 = 'iso-8601'
+STD_DURATION = 'standard'


It turns out that this value isn't used anywhere; as long as format is set to anything else but ISO_8601, we end up with 'standard' behavior.

That's probably not intended. Can this be correct? Let's also add a test case for the behavior with an unknown format value.

That value was placed just to be a placeholder for the default format. It may just be set to None in settings.py. I added the "standard" after this conversation just to improve readability. The format-as-ISO may also be a flag, however that pattern may be harder to read and would add a non-uniformity with other time-related fields.

It is true that django itself does not provide much flexibility with duration representation, which also comes from stdlib timedelta which has no implementation with to-str or from-str like date/time does.

Which way do you prefer for completing this?

browniebroke · 2025-04-28T11:38:10Z

rest_framework/fields.py

@@ -1351,9 +1351,11 @@ class DurationField(Field):
        'overflow': _('The number of days must be between {min_days} and {max_days}.'),
    }

-    def __init__(self, **kwargs):
+    def __init__(self, format=empty, **kwargs):


We should disallow specifying this as positional argument: DurationField('iso-8601') (we don't want that).

All other options come from the **kwargs...

browniebroke · 2025-04-28T11:43:52Z

tests/test_fields.py

+    valid_inputs = {}
+    invalid_inputs = {}


Would be nice to add test coverage for parsing duration in ISO format, as I would expect it to work both ways

Currently the parse_duration method of django, which is used inside DurationField, already parses ISO-8601 format and the format specific to postgres by its own.

This could be added to the TestDurationField testcase.

Where do you prefer these tests to be added?

PS:
From this point of view the current error message "Use one of these formats instead: [DD] [HH:[MM:]]ss[.uuuuuu]" is not (completely) right, however changing it would be a breaking change.

I'm not sure where that an extension of this error message should be considered a breaking change; the wording "one of" already hints that the application should not rely the one specific format being given in the message.

tests/test_fields.py

browniebroke · 2025-04-28T11:47:48Z

docs/api-guide/fields.md

+* `format` - A string representing the output format.  If not specified, this defaults to the same value as the `DURATION_FORMAT` settings key, which will be `'standard'` unless set. Setting to a format string indicates that `to_representation` return values should be coerced to string output. Format strings are described below. Setting this value to `None` indicates that Python `timedelta` objects should be returned by `to_representation`. In this case the date encoding will be determined by the renderer.
 * `max_value` Validate that the duration provided is no greater than this value.
 * `min_value` Validate that the duration provided is no less than this value.

+#### `DurationField` format strings
+Format strings may either be the special string `'iso-8601'`, which indicates that [ISO 8601][iso8601] style intervals should be used (eg `'P4DT1H15M20S'`), or the special string `'standard'`, which indicates that Django interval format `'[DD] [HH:[MM:]]ss[.uuuuuu]'` should be used (eg: `'4 1:15:20'`).


Good point, how about format_name?

Co-authored-by: Bruno Alla <[email protected]>

tomchristie reviewed Jun 24, 2022

View reviewed changes

stale bot added the stale label Oct 16, 2022

stale bot closed this Nov 1, 2022

auvipy reopened this Jun 4, 2024

stale bot removed the stale label Jun 4, 2024

sevdog force-pushed the duration-format branch from 9d506ba to 9713c1d Compare June 5, 2024 09:28

sevdog marked this pull request as ready for review June 5, 2024 09:33

sevdog force-pushed the duration-format branch 2 times, most recently from 295a462 to 70751a5 Compare July 8, 2024 12:41

auvipy requested changes Jul 9, 2024

View reviewed changes

sevdog force-pushed the duration-format branch from 70751a5 to 35398b3 Compare July 9, 2024 16:36

auvipy requested a review from a team July 10, 2024 09:24

peterthomassen previously approved these changes Jul 10, 2024

View reviewed changes

docs/api-guide/fields.md Show resolved Hide resolved

sevdog force-pushed the duration-format branch from 35398b3 to bcc4ebf Compare July 11, 2024 06:13

stale bot added the stale label Apr 26, 2025

auvipy removed the stale label Apr 27, 2025

auvipy reviewed Apr 27, 2025

View reviewed changes

Allow format duration as ISO-8601

5723ba1

sevdog dismissed peterthomassen’s stale review via 5723ba1 April 28, 2025 10:07

sevdog force-pushed the duration-format branch from bcc4ebf to 5723ba1 Compare April 28, 2025 10:07

peterthomassen requested changes Apr 28, 2025

View reviewed changes

browniebroke reviewed Apr 28, 2025

View reviewed changes

auvipy and others added 2 commits April 29, 2025 12:28

Update tests/test_fields.py

c3efa55

Co-authored-by: Bruno Alla <[email protected]>

Update tests/test_fields.py

4377b08

Co-authored-by: Bruno Alla <[email protected]>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

DurationField output format #8532

DurationField output format #8532

sevdog commented Jun 24, 2022

tomchristie Jun 24, 2022

sevdog Jun 24, 2022

tomchristie Jun 24, 2022

tomchristie Jun 24, 2022

sevdog Jun 24, 2022

tomchristie commented Jun 24, 2022

stale bot commented Oct 16, 2022

pietzschke commented Jun 3, 2024

pietzschke commented Jun 5, 2024

sevdog commented Jun 5, 2024

auvipy left a comment

sevdog commented Jul 9, 2024

auvipy commented Jul 11, 2024

stale bot commented Apr 26, 2025

ulgens commented Apr 27, 2025

auvipy left a comment

sevdog commented Apr 28, 2025

peterthomassen left a comment

peterthomassen Apr 28, 2025

browniebroke Apr 28, 2025

peterthomassen Apr 28, 2025

sevdog Apr 28, 2025

peterthomassen Apr 28, 2025

sevdog Apr 28, 2025

browniebroke Apr 28, 2025

browniebroke Apr 28, 2025

sevdog Apr 28, 2025

peterthomassen Apr 28, 2025

browniebroke Apr 28, 2025

	Format strings may either be the special string `'iso-8601'`, which indicates that [ISO 8601][iso8601] style intervals should be used (eg `'P4DT1H15M20S'`), or the special string `'standard'`, which indicates that Django interval format `'[DD] [HH:[MM:]]ss[.uuuuuu]'` hould be used (eg: `'4 1:15:20'`).
	Format strings may either be the special string `'iso-8601'`, which indicates that [ISO 8601][iso8601] style intervals should be used (eg `'P4DT1H15M20S'`), or the special string `'standard'`, which indicates that Django interval format `'[DD] [HH:[MM:]]ss[.uuuuuu]'` should be used (eg: `'4 1:15:20'`).


		#### DURATION_FORMAT

		A format string that should be used by default for rendering the output of `DurationField` serializer fields. If `None`, then `DurationField` serializer fields will return Python `timedelta` objects, and the duration encoding will be determined by the renderer.

	elif isinstance(obj, datetime.timedelta):
	return str(obj.total_seconds())

DurationField output format #8532

Are you sure you want to change the base?

DurationField output format #8532

Conversation

sevdog commented Jun 24, 2022

Description

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

tomchristie commented Jun 24, 2022

stale bot commented Oct 16, 2022

pietzschke commented Jun 3, 2024

pietzschke commented Jun 5, 2024

sevdog commented Jun 5, 2024

auvipy left a comment

Choose a reason for hiding this comment

sevdog commented Jul 9, 2024

auvipy commented Jul 11, 2024

stale bot commented Apr 26, 2025

ulgens commented Apr 27, 2025

auvipy left a comment

Choose a reason for hiding this comment

sevdog commented Apr 28, 2025

peterthomassen left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment