Attention

This is the documentation for HARP Proxy, actually published as an early preview. Both the software and documentation are a work in progress, and although we already use it on various production servers, they may contain inaccuracies, typographical errors, huge mistakes and empty pages. We work hard to eradicate all mistakes and implement stuff, but it is a long and tedious process. We appreciate your patience and understanding. Of course, any help will be greatly appreciated.

HttpClient¶

The harp_apps.http_client application implements the core HTTP client features. It uses caching to store responses and avoid making the same request multiple times, improving the efficiency of your application.

The caching mechanism is implemented using Hishel a powerful caching library.

Overview¶

The HTTP client provides efficient and configurable HTTP request handling with caching capabilities. It is designed to be integrated seamlessly into the harp framework.

Features¶

Caching: Reduces redundant network calls by storing responses.
Configurable Timeouts: Allows setting custom timeout values for requests.
Flexible Cache Settings: Offers options to configure cacheable methods and status codes.

Loading¶

The HTTP client application is loaded by default when using the harp start command.

Configuration¶

Below is an example configuration for the HTTP client:

http_client:
  # HTTP timeout (default to `harp.settings.DEFAULT_TIMEOUT`)
  timeout: 10.0

  # Cache configuration (optional)
  cache:
    enabled: true  # Set to false to disable cache entirely (optional)

    controller:  # Override the cache controller definition (optional)
      cacheable_methods: [GET, HEAD]  # Which HTTP methods should be cacheable? (default: [GET, HEAD])
      force_cache: true  # or any other hishel.Controller option ....

You can refer to hishel.Controller documentation for all available options.

timeout: Specifies the request timeout duration in seconds (default: 30 seconds).
cache: Configuration for caching behavior.
- disabled: Boolean flag to enable or disable caching.
- controller: Configuration for controller settings.
  - allow_stale: Boolean flag to allow serving stale cache data when the cache is expired (default: False).
  - allow_heuristics: Boolean flag to allow heuristic caching (default: False).
  - cacheable_methods: List of HTTP methods that can be cached (e.g., GET).
  - cacheable_status_codes: List of HTTP status codes that can be cached (e.g., 200, 300).

Internal Implementation¶

The internal implementation leverages the following classes:

ControllerSettings
CacheSettings
HttpClientSettings

Full example¶

http_client:
  # HTTP timeout (default to `harp.settings.DEFAULT_TIMEOUT`)
  timeout: 10.0

  # Customize the httpx transport layer (optional)
  transport:
    # This is the default, only set if you want to use a custom transport. Most users don't need to set this.
    "@type": "httpx:AsyncHTTPTransport"

  # Cache configuration (optional, enabled by default)
  cache:
    # Set to false to disable cache entirely
    enabled: true

    # Override the cache controller definition (optional)
    controller:
      # This is the default, only set if you want to use a custom controller.
      "@type": "hishel:Controller"

      # You can configure anything the hishel cache controller would accept as a keyword argument.
      # See https://hishel.com/advanced/controllers/ for more information.

      # Should stale cache entries be returned if the cache is being refreshed? (default: true)
      allow_stale: true

      # Should heuristics be used to determine cache expiration? (default: true)
      allow_heuristics: true

      # Which HTTP methods should be cacheable? (default: [GET, HEAD])
      cacheable_methods: [GET, HEAD]

      # Which status codes should be cacheable? (default: see `hishel.HEURISTICALLY_CACHEABLE_STATUS_CODES`)
      cacheable_status_codes: [200, 301, 308]

    # Customize the cache transport layer (optional). The cache transport layer is a decorator arount httpx transport
    # layer extending the base http client features with caching.
    transport:
      # This is the default, only set if you want to use a custom cache transport.
      "@type": "hishel:AsyncCacheTransport"

      # If your hishel compatible transport class take more kwargs, you can pass more stuff here.
      # See https://hishel.com/userguide/#clients-and-transports

    # Customize the hishel cache storage (optional)
    # Please note that we plan to allow to share other harp storages here in the future.
    storage:
      # This is the default, only set if you want to use a custom cache storage.
      "@type": "hishel:AsyncFileStorage"

      # If your hishel compatible storage class take more kwargs, you can pass more stuff here.
      # See https://hishel.com/advanced/storages/