Skip to content

Commit d06001f

Browse files
jennifer-richardsrjsparkshollowaykesararudimatz
authored
feat: RPC modernization APIs (#9631)
* feat: Add rpc_person API call * chore: Add OpenAPI yaml for RPC API * feat: api endpoint for docs submitted to the rpc * chore: remove some debug * feat: api for created demo people * feat: optimize fetching persons * feat: api for creating demo drafts (#6402) * fix: Typo in rpcapi.yaml * refactor: Allow existing Person in create_demo_person (#6398) * fix: include pubreq docevents in demo drafts (#6404) * fix: Minor API fixes (#6405) * chore: Document 404 from rpcapi get_person_by_id * feat: adding rev to demo doc creation (#6425) * feat: adding rev to demo doc creation * fix: remove attempt to control required * refactor: Replace api token checks with decorator (#6434) * feat: Add @requires_api_token decorator * refactor: Use @requires_api_token * refactor: Tweak api token endpoints This might be drifting from the design intent, but at least uses the defined endpoint values. Further cleanup may well be needed. * refactor: Improve usability of @requires_api_token * feat: get_draft_by_id api call (#6446) * fix: construct the cdn photo url correctly (#6491) * chore: restructure the rpc api (#6505) * fix: authenticate person api correctly * chore: Fix plain_name lookup * feat: subject_id -> Person api call (#6566) * feat: subject_id -> Person api call * doc: Add error responses to openapi spec * feat: rpc api drafts by names (#6853) * feat: get stream rfcs were first published into (#7814) * feat: get stream rfcs were first published into * chore: black * fix: deal with the case of no DocHistory having a stream * fix: return "none" from drafts_by_names if a draft has a stream of None (#7863) * feat: API changes needed for "real" draft imports (#7877) * style: black * fix: submitted as DateTime, not Date * fix: pk -> id in API * feat: rpc_draft source_format * feat: add shepherd to rpc_draft() api * fix: "unknown" src fmt instead of error * feat: add intended_std_level to api * refactor: blank, not None, for shepherd / std_level * style: black * fix: typo in drafts_by_names() (#7912) * feat: support for building rfc authors (#7983) * feat: support for building rfc authors * chore: copyrights * feat: api to gather draft authors (#8126) * feat: api to gather draft authors * chore: black * ci: tag feature branch release images * chore: fix git nonsense * feat: API for RFC metadata fetch/filtering (#8291) * feat: RFC API for rfceditor website (WIP) * feat: pagination + ordered RFC index * feat: filter by publication date * feat: stream + stream filtering * feat: DOI * feat: group/area for RFCs * feat: group/area filtering * feat: result sorting * refactor: send rfc number, not name * feat: search rfc title/abstract * style: Black * feat: add 'status' field * feat: filter by 'status' * style: remove redundant parentheses * feat: add updated_by/obsoleted_by fields * feat: add 'abstract' to rpc api * chore: fix unused/duplicate imports * chore: fix mypy lint * chore: unused import * feat: retrieve single rfc, including text (#8346) * feat: retrieve single rfc Use RFC number instead of doc PK as id * feat: include text in single-rfc response * chore: drop doc id from api response * fix: many=True for identifiers (#8425) * feat: add more rfc api fields (many stubs) * chore: adding postscript (ps) to rfc meta serializer (#8560) * fix: acknowledge not-issued in RfcStatusSlugT * feat: Add API call to get references * fix: Filter drafts * chore: Optimize the data query * test: Test for norminative references API call * chore: Fix typos in tests * chore: Fix typos * refactor: Separate demo logic (#8937) * refactor: Separate demo logic * chore: Skip tests * fix: trailing slashes for all rpc api endpoints (#8940) * chore: Add RPC references API call to OpenAPI spec (#8941) * chore: Remove line noise * chore: Add RPC references API call to OpenAPI spec * chore: Update rpcapi.yaml Co-authored-by: Robert Sparks <rjsparks@nostrum.com> --------- Co-authored-by: Robert Sparks <rjsparks@nostrum.com> * fix: Fix OpenAPI spec errors (#8943) * chore: Remove more line noise * fix: Fix OpenAPI spec errors * feat: include picture URL in rpc_person API (#9009) * feat: person search endpoint (#9062) * feat: person search endpoint * refactor: address review comments * improved naming of operation/components in API schema * reused Person schema component * added serializers_rpc.py * chore: RpcPersonSerializer -> PersonSerializer Better matches the hand-written schema. * fix: search for entire term, not word-by-word * fix: only look at name/plain in search Including ascii / ascii_short might be useful eventually, but since we only show plain_name in the response it can cause confusing results. By the same reasoning we could remove email__address as well, but that's useful and I expect we'll include email addresses in our response soon anyway. * refactor: reimplement purple API in django-rest-framework (#9097) * refactor: rpc_person -> PersonViewSet * refactor: rpc_subject_person -> SubjectPersonView * refactor: rpc_persons -> RpcPersonsView * refactor: move get_persons into PersonViewSet Changes the interface to return a list of Persons instead of a map from ID to name. * refactor: rpc_draft -> DraftViewSet * refactor: drafts_by_names -> DraftsByNameView * refactor: submitted_to_rpc -> DraftViewSet * refactor: rfc_original_stream -> RfcViewSet * refactor: rpc demo APIs -> viewset * refactor: get_draft_refs -> DraftViewSet * refactor: persons_by_email -> PersonViewSet * refactor: rfc_authors -> RfcViewSet * refactor: draft_authors -> DraftViewSet * refactor: avoid \x00 in regex validator Gets turned into a literal nul somewhere in the process of generating a schema and building a Python client for purple. This has the same effect but avoids the nul. * fix: missing arg on references() action * style: ruff, remove unused imports * style: ruff ruff * chore: remove rpcapi.yaml * refactor: move API to /api/purple Side effect is that the purple API client is named PurpleApi instead of RpcApi. * fix: get_draft_authors returns DraftWithAuthors * fix: distinguish CharField flavors * fix: no serializer validators for draft name/title This prevents at least one existing draft from being looked up. * fix: get_draft_authors works with str, not int * Revert "refactor: avoid \x00 in regex validator" This reverts commit 63f40cf * Revert "Revert "refactor: avoid \x00 in regex validator"" (#9111) This reverts commit d8656f4. * ci: only migrate blobdb if it is configured * feat: add email/url to purple person API (#9127) * feat: expose consensus in submission api * feat: subseries api for red (#9556) * refactor: central def of subseries doc types * feat: subseries doc API * refactor: optimize queries via prefetch Reduced 4500 to 18 queries * chore: remove debug * fix: fix serialization of draft field * refactor: clean up prefetch a bit * feat: filter by subseries type * fix: restore max_limit for RFC pagination * feat: add subseries+stub titlepage_name to rfc serializer (#9569) * feat: add subseries to RfcMetadataSerializer * feat: titlepage_name for RfcAuthorSerializer Always blank for now * chore: update copyrights * refactor: use py3.12 typing syntax * fix: renumber migrations * feat: add consensus on FullDraftSerializer * feat: add type field in serializer * feat: tag subseries API endpoints for purple (#9763) * feat: tag subseries API endpoints for purple * ruff * feat: change slugs/names (#9778) * change slugs/names * change slug names * fix: update RfcStatusSlugT * fix: remove double tag (#9787) This was meant to include the API call in both purple and red API clients. It seems this does not work, at least with some generators. Need to investigate further, but we should be able to work around it. * feat: rfc authors (#9937) * feat: rfc authors * fix: distinct rfc search results * fix: include titlepage_name in author name searches * fix: add is_editor to rfcauthor model. Adjust FK on_delete. Tweak admin. * fix: renumber migration * refactor: realistic titlepage_name in factory * refactor: comment + rename iteration var * chore: bump copyright year * chore: remove country from RfcAuthor Not planning to track this * refactor: make blank=False explicit * chore: remove country from admin * fix: author_list() for type=rfc * fix: blankable RfcAuthor.person * feat: RfcAuthor-aware document_json() * feat: limit docs to rfcs for RfcAuthor admin * test: document_json authors * fix: use author_names() for get_document_emails() * feat: suggest affiliation based on RfcAuthor * chore: revert "remove country from RfcAuthor" This reverts commit 3044d10. * chore: revert "remove country from admin" This reverts commit 2088793. * feat: use rfcauthors for nomcom eligibility (#9629) * refactor: author-based eligibility cleanup * feat: use rfcauthor recs for nomcom eligbility * chore: remove commented code * refactor: factor out helper for testing * test: test_get_qualified_author_queryset * fix: restore a necessary import * test: fix test_elig_by_author * test: fix test_decorate_volunteers_with_qualifications * test: add comment * fix: drop test for draft-rfceditor state Attempted to limit to drafts literally in the queue, but was not a valid check when looking back in time. As a practical matter, the test is not necessary. * fix: exclude double counting, not rfc pub state * test: update test --------- Co-authored-by: Robert Sparks <rjsparks@nostrum.com> * fix: renumber migrations * feat: basic RfcAuthor API (#9940) * refactor: move get_rfc_authors API URL * refactor: drop format_suffixes from router Creates a bunch of API endpoints we have no intention of ever using * feat: RfcAuthor API (WIP) * fix: remove debug code * style: remove stray whitespace * fix: authors API for xfer (#9961) * fix: partial implementation of rfc authors() Does not handle RfcAuthor instances where person is None yet. * refactor: authors -> bulk_authors for URL consistency * fix: add OpenApi param definition doc_id (#9962) * fix: add OpenApi param definition doc_id * add schema def for person_id * feat: RFC publication API (#9975) * feat: API to publish RFC (WIP) Incomplete and in need of refactoring, but publishes an RFC. * feat: group / formal_languages from draft * feat: allow optional formal_languages via API Could do the same with group, but not clear it would ever be used. * feat: fill in overrides/updates * feat: subseries membership * fix: tolerate race to create related docs * fix: wrap pub in a transaction * feat: prevent re-publishing draft as RFC * chore: remove stale code * chore: remove debug * feat: RFC file upload API (WIP) Checkpointing progress before going further. * feat: specify RFC, validate file exts * feat: move uploaded files into place * feat: add replace option * fix: add rest of replace option * feat: handle ad/group more consistently * chore: remove inadvertent change * chore: drop external_url, get note from draft * refactor: clarify default value logic * refactor: ID obsoletes/updates by number * fix: handle draft-stream-editorial * feat: split unknown vs already published draft error (#10011) * feat: use RfcAuthor for red API (#10014) * refactor: combine redundant serializers * feat: edit authors via RFC update API * fix: remove all None authors, not just first (#10044) * refactor: extract update_rfcauthors() method * feat: EditedRfcAuthorsDocEvent * refactor: reduce RfcAuthor instance churn * feat: create RfcAuthor edit DocEvents * feat: handle DocumentAuthor->RfcAuthor updates * refactor: reduce code duplication * fix: remove leftover import * chore: relabel titlepage_name in DocEvent desc * feat: transaction * chore: make RfcAuthorViewset read-only This can perhaps go away entirely * style: undo accidental whitespace change * fix: actual RFC file extensions/locations + tests (#10131) * chore: update list of rfc file exts * refactor: _destination() helper * fix: .notprepped.xml -> prerelease/ subdir * refactor: better prefixed DRF Routers * test: fix references API test * test: test notify_rfc_published * test: test upload_rfc_files * chore: remove unused imports * chore: remove unused imports * chore: add a todo * fix: find pubreq event for editorial stream * chore: remove obsolete rpc demo API * chore: clean up outdated comments * feat: api key for red api (#10232) * fix: avoid over-return (#10231) * fix: avoid over-return * chore: undo accidental commit This is a separate bug fix on main; let it come in from there. * refactor: authors() -> author_persons() (#10237) * refactor: authors() -> author_persons() * refactor: select_related() a couple more places * refactor: update uses of Document.authors() * chore: remove debug * fix: typo * fix: mypy lint and minor bugs; mypy->1.11.2 (#10249) * fix: doc property authors needs refactoring (#10250) * fix: doc property needs refactoring * fix: set source for author fields * chore: comment * chore: fix random typo --------- Co-authored-by: Jennifer Richards <jennifer@staff.ietf.org> * chore: remove completed todo, add comment * chore(dev): accept devtoken for red api --------- Co-authored-by: Robert Sparks <rjsparks@nostrum.com> Co-authored-by: Matthew Holloway <matthew@staff.ietf.org> Co-authored-by: Kesara Rathnayake <kesara@fq.nz> Co-authored-by: Rudi Matz <rudi@staff.ietf.org>
1 parent 79cb013 commit d06001f

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

52 files changed

+2945
-315
lines changed

dev/build/migration-start.sh

Lines changed: 6 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,11 @@
33
echo "Running Datatracker migrations..."
44
./ietf/manage.py migrate --settings=settings_local
55

6-
echo "Running Blobdb migrations ..."
7-
./ietf/manage.py migrate --settings=settings_local --database=blobdb
6+
# Check whether the blobdb database exists - inspectdb will return a false
7+
# status if not.
8+
if ./ietf/manage.py inspectdb --database blobdb > /dev/null 2>&1; then
9+
echo "Running Blobdb migrations ..."
10+
./ietf/manage.py migrate --settings=settings_local --database=blobdb
11+
fi
812

913
echo "Done!"

dev/deploy-to-container/settings_local.py

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -71,11 +71,11 @@
7171

7272
DE_GFM_BINARY = '/usr/local/bin/de-gfm'
7373

74-
# No real secrets here, these are public testing values _only_
7574
APP_API_TOKENS = {
76-
"ietf.api.views.ingest_email_test": ["ingestion-test-token"]
75+
"ietf.api.red_api" : ["devtoken", "redtoken"], # Not a real secret
76+
"ietf.api.views.ingest_email_test": ["ingestion-test-token"], # Not a real secret
77+
"ietf.api.views_rpc" : ["devtoken"], # Not a real secret
7778
}
7879

79-
8080
# OIDC configuration
8181
SITE_URL = 'https://__HOSTNAME__'

docker/configs/settings_local.py

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -100,3 +100,8 @@
100100
bucket_name=f"{storagename}",
101101
),
102102
}
103+
104+
APP_API_TOKENS = {
105+
"ietf.api.red_api" : ["devtoken", "redtoken"], # Not a real secret
106+
"ietf.api.views_rpc" : ["devtoken"], # Not a real secret
107+
}

ietf/api/routers.py

Lines changed: 20 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -3,14 +3,29 @@
33
from django.core.exceptions import ImproperlyConfigured
44
from rest_framework import routers
55

6-
class PrefixedSimpleRouter(routers.SimpleRouter):
7-
"""SimpleRouter that adds a dot-separated prefix to its basename"""
6+
7+
class PrefixedBasenameMixin:
8+
"""Mixin to add a prefix to the basename of a rest_framework BaseRouter"""
89
def __init__(self, name_prefix="", *args, **kwargs):
910
self.name_prefix = name_prefix
1011
if len(self.name_prefix) == 0 or self.name_prefix[-1] == ".":
1112
raise ImproperlyConfigured("Cannot use a name_prefix that is empty or ends with '.'")
1213
super().__init__(*args, **kwargs)
1314

14-
def get_default_basename(self, viewset):
15-
basename = super().get_default_basename(viewset)
16-
return f"{self.name_prefix}.{basename}"
15+
def register(self, prefix, viewset, basename=None):
16+
# Get the superclass "register" method from the class this is mixed-in with.
17+
# This avoids typing issues with calling super().register() directly in a
18+
# mixin class.
19+
super_register = getattr(super(), "register")
20+
if not super_register or not callable(super_register):
21+
raise TypeError("Must mixin with superclass that has register() method")
22+
super_register(prefix, viewset, basename=f"{self.name_prefix}.{basename}")
23+
24+
25+
class PrefixedSimpleRouter(PrefixedBasenameMixin, routers.SimpleRouter):
26+
"""SimpleRouter that adds a dot-separated prefix to its basename"""
27+
28+
29+
class PrefixedDefaultRouter(PrefixedBasenameMixin, routers.DefaultRouter):
30+
"""DefaultRouter that adds a dot-separated prefix to its basename"""
31+

0 commit comments

Comments
 (0)