~cloutier/bird.makeup#23: 
Setting up new instance of BirdMakeup (Object reference not set to an instance of an object.)

Hello,

Attempting to setup a fresh new instance of this using Docker-Compose.

The site loads, but whenever I submit a Twitter handle I get the following error:

2023-07-20 12:57:28 fail: Microsoft.AspNetCore.Diagnostics.ExceptionHandlerMiddleware[1] 2023-07-20 12:57:28 An unhandled exception has occurred while executing the request. 2023-07-20 12:57:28 System.NullReferenceException: Object reference not set to an instance of an object. 2023-07-20 12:57:28 at BirdsiteLive.Controllers.UsersController.Index(String id) in /home/build/bird.makeup/src/BirdsiteLive/Controllers/UsersController.cs:line 133 2023-07-20 12:57:28 at Microsoft.AspNetCore.Mvc.Infrastructure.ActionMethodExecutor.TaskOfIActionResultExecutor.Execute(ActionContext actionContext, IActionResultTypeMapper mapper, ObjectMethodExecutor executor, Object controller, Object[] arguments) 2023-07-20 12:57:28 at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.g__Awaited|12_0(ControllerActionInvoker invoker, ValueTask`1 actionResultValueTask) 2023-07-20 12:57:28 at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.g__Awaited|10_0(ControllerActionInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted) 2023-07-20 12:57:28 at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Rethrow(ActionExecutedContextSealed context) 2023-07-20 12:57:28 at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Next(State& next, Scope& scope, Object& state, Boolean& isCompleted) 2023-07-20 12:57:28 at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.g__Awaited|13_0(ControllerActionInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted) 2023-07-20 12:57:28 at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.g__Awaited|25_0(ResourceInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted) 2023-07-20 12:57:28 at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.Rethrow(ResourceExecutedContextSealed context) 2023-07-20 12:57:28 at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.Next(State& next, Scope& scope, Object& state, Boolean& isCompleted) 2023-07-20 12:57:28 at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.g__Awaited|20_0(ResourceInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted) 2023-07-20 12:57:28 at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.g__Awaited|17_0(ResourceInvoker invoker, Task task, IDisposable scope) 2023-07-20 12:57:28 at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.g__Awaited|17_0(ResourceInvoker invoker, Task task, IDisposable scope) 2023-07-20 12:57:28 at Microsoft.AspNetCore.Routing.EndpointMiddleware.g__AwaitRequestTask|6_0(Endpoint endpoint, Task requestTask, ILogger logger) 2023-07-20 12:57:28 at Microsoft.AspNetCore.Authorization.AuthorizationMiddleware.Invoke(HttpContext context) 2023-07-20 12:57:28 at Microsoft.AspNetCore.Diagnostics.ExceptionHandlerMiddlewareImpl.g__Awaited|8_0(ExceptionHandlerMiddlewareImpl middleware, HttpContext context, Task task)

Status
REPORTED
Submitter
~peanutlasko
Assigned to
No-one
Submitted
1 year, 2 days ago
Updated
8 months ago
Labels
No labels applied.

~eric-pierce 1 year, 2 days ago

~peanutlasko would you paste the birdmakeup part of your docker-compose.yml here as well? Also, has anything populated into the databases you set up?

~peanutlasko 1 year, 2 days ago*

server:

    image: cloutier/bird.makeup:latest
    restart: always
    hostname: birdmakeup.local
    container_name: ${COMPOSE_PROJECT_NAME}_birdmakeup
    environment:
        - Instance:Domain=birdmakeup.$DOMAINNAME2
        - Instance:Name=birdmakeup.$DOMAINNAME2
        - Instance:AdminEmail=$EMAIL
        - Instance:ResolveMentionsInProfiles=true
        - Instance:ParallelTwitterRequests=20
        - Instance:ParallelFediverseRequests=20
        #- Instance:PublishReplies=true
        - Db:Type=postgres
        - Db:Host=birdmakeupdb.local
        - Db:Name=birdmakeupdb
        - Db:User=$DBUSERNAME
        - Db:Password=$DBPASSWORD
        #- Twitter:ConsumerKey=$APIKEY
        #- Twitter:ConsumerSecret=$APIKEYSECRET
        - Moderation:FollowersWhiteListing=${WHITELIST}
        #- Logging:insights
        - PUID=$PUID
        - PGID=$PGID
        - TZ=$TZ
    networks:
        - macdocker_proxy1
        - birdsitelivenetwork
    # volumes:
    #     - type: bind
    #       source: $HOME_DIR/$COMPOSE_PROJECT_NAME/birdmakeup/key.json
    #       target: /app/key.json
    # #ports:
    #    - "5000:80"
    depends_on:
        - db
    labels:
        - 'com.centurylinklabs.watchtower.enable=true'
        - 'traefik.docker.network=macdocker_proxy1'
        #enabled
        - 'traefik.enable=true'
        ## TCP Routers
        - 'traefik.http.routers.birdsitelive-rtr.entrypoints=https'
        - 'traefik.http.routers.birdsitelive-rtr.rule=Host(`birdmakeup.$DOMAINNAME2`)'
        - 'traefik.http.routers.birdsitelive-rtr.tls=true'
        #middleware
        - 'traefik.http.routers.birdsitelive-rtr.middlewares=chain-no-auth@file'
        #- 'traefik.http.routers.birdsitelive-rtr.middlewares=chain-basic-auth@file'
        #- 'traefik.http.routers.birdsitelive-rtr.middlewares=chain-oauth@file'
        #defines router and port
        - 'traefik.http.routers.birdsitelive-rtr.service=birdsitelive-svc'
        - 'traefik.http.services.birdsitelive-svc.loadbalancer.server.port=${BIRDSITE_PORT}'

db:

 image: postgres:15
 restart: always
 hostname: birdmakeupdb.local
    container_name: ${COMPOSE_PROJECT_NAME}_birdmakeupdb
    environment:
        - POSTGRES_USER=$DBUSERNAME
        - POSTGRES_PASSWORD=$DBPASSWORD
        - POSTGRES_DB=birdmakeupdb
    networks:
        - birdsitelivenetwork
    volumes:
        - postgresbirdmakeup:/var/lib/postgresql/data
    labels:
        - 'com.centurylinklabs.watchtower.enable=true'
        - 'com.centurylinklabs.watchtower.monitor-only=true'

~peanutlasko 1 year, 2 days ago*

edit: nvm got it

~eric-pierce 1 year, 1 day ago*

~peanutlasko it looks like the error is being thrown because birdmakeup is trying to reference a user that doesn't exist:

2023-07-20 12:57:28 at BirdsiteLive.Controllers.UsersController.Index(String id) in /home/build/bird.makeup/src/BirdsiteLive/Controllers/UsersController.cs:line 133 

https://git.sr.ht/~cloutier/bird.makeup/tree/master/item/src/BirdsiteLive/Controllers/UsersController.cs#L133

Do you have data populated in the twitter_users table in your postgres database? Here's an example of what that data should look like: https://todo.sr.ht/~cloutier/bird.makeup/4#event-234718. If that table is empty, populating it with data for a single user to follow may resolve this.

Do you have data populated in the twitter_users table in your postgres database? Here's an example of what that data should look like: https://todo.sr.ht/~cloutier/bird.makeup/4#event-234718. If that table is empty, populating it with data for a single user to follow may resolve this.

It looks like ~cloutier pushed some updates to address the missing data in tables, but I haven't heard any news as to how that works. if you're missing data in those tables the fix may not have resolved it.

One other thing to try is to change the image you're using from the one on dockerhub, to the one ~rursache hosted on github. The github image uses version 7 of aspnet and the dotnet SDK instead of version 6 used in the dockerhub version. You'd just need to update the image piece of your docker-compose to be:

image: ghcr.io/rursache/bird.makeup:latest

instead of

image: cloutier/bird.makeup:latest

Note: don't update to the github hosted image if you run on an arm64 device like a raspberry pi as that image only supports x86 architecture. I did open a patchset for the changes in ~rursache's Dockerfile but it doesn't look like ~cloutier has looked or merged it.

~rursache 1 year, 1 day ago

~peanutlasko instructions of how to populate the empty table are also available here: https://github.com/rursache/bird.makeup/issues/3

if you solved your issue leave explicit details on how you did it so others will benefit too.

~peanutlasko 1 year, 1 day ago

Thanks ~rusache, currently stuck on how to get pgadmin working in a docker container. Frustrating to say the least

~rursache 1 year, 1 day ago

~peanutlasko

docker run -d \
  --name pgadmin \
  --restart unless-stopped \
  -p 2210:80 \
  -e 'PGADMIN_DEFAULT_EMAIL=YOUR@EMAIL.TLD' \
  -e 'PGADMIN_DEFAULT_PASSWORD=UNIQUEPASS' \
  -v /home/USER/.pgadmin:/var/lib/pgadmin \
  dpage/pgadmin4:latest

~peanutlasko 1 year, 1 day ago

Eric, I tried using the github version you posted on a clean DB and doesn't appear to work either.

~eric-pierce 1 year, 1 day ago

~peanutlasko you can also query from the command line without pgadmin https://stackoverflow.com/questions/71104319/postgresql-select-from-table-inside-docker-container-bash if getting that image up and running is tough. No UI so you have to use SQL directly but may be faster/easier.

You'd run the following after replacing the parameters with the values you're using in your .env file

sudo docker exec -it ${COMPOSE_PROJECT_NAME}_birdmakeupdb psql -U $DBUSERNAME birdmakeupdb

Then when that gives you a SQL prompt just type in

select * from twitter_users;

and see what it returns. If nothing comes back and you need to insert a row you'd have to use a SQL insert statement, which I can help with if it comes to that.

~peanutlasko 1 year, 1 day ago

So I got the pg4admin running, I've ran the queries and both my twitter_users and followers tables are empty. I've entered in the data, but as soon as i boot up my birdmakeup container, the tables get wiped and im back at 0 again with the same error as before. This is beyond frustrating.

~peanutlasko 1 year, 1 day ago*

Also noted my followers table doesnt have a column for followingssyncstatus nor does my twitter_users have a asttweetsynchronizedforallfollowersid column

~eric-pierce 1 year, 1 day ago

~peanutlasko yes fully feel the frustration, it seems like the setup may be broken. I'm going to try setting up a fresh install and see if I can re-create what you're seeing, and hopefully solve.

~eric-pierce 1 year, 1 day ago*

~peanutlasko Ok I was able to re-create the issues you're seeing on a fresh install, and got everything working. The problem is resolved by populating the data below in the two tables (followers, twitter_users).

May be worth doing a fresh install rather than trying to resolve your current setup, so stop and remove the container, delete the database and birdmakeup user from postgres using pgadmin.

After you've stopped and remove the bmu container, and removed the db/user from postgres, do the following:

  1. Create BMU user in postgres
  2. Give superuser privileges
  3. Create the birdmakeup database, and set it to be owned by your birdmakeup user

Start up the container, you should get a working UI at your birdmakeup domain

In pgadmin, paste the following values into the two relevant tables:

followers:

id	followings	acct	host	inboxroute	sharedinboxroute	actorid	postingerrorcount
1	{1}	peanutlasko	mas.to	/users/peanutlasko/inbox	/inbox	https://mas.to/users/peanutlasko	0

twitter_users - NOTE that the lastsync timestamp has spaces in it (2023-07-21 14:54:07.16) and the following 0 is for the fetchingerrorcount. Leave fediverseaccount NULL

id	acct	lasttweetpostedid	lastsync	fetchingerrorcount	twitteruserid	fediverseaccount
1	npr	1646138106595139584	2023-07-21 14:54:07.16	0	5392522

Then restart the bmu container and it should be working

~peanutlasko 1 year, 1 day ago

better now! Thanks a lot for all this help Eric. So it appears to be working somewhat - I can log into my Mastodon instance and follow users from twitter. However, if I visit the actual birdmakeup..com that I self host, then I enter in any Twitter handle - @MKBHD for example, I still get the error:

2023-07-21 15:40:02 fail: Microsoft.AspNetCore.Diagnostics.ExceptionHandlerMiddleware[1] 2023-07-21 15:40:02 An unhandled exception has occurred while executing the request. 2023-07-21 15:40:02 System.NullReferenceException: Object reference not set to an instance of an object. 2023-07-21 15:40:02 at BirdsiteLive.Controllers.UsersController.Index(String id) in /src/BirdsiteLive/Controllers/UsersController.cs:line 133 2023-07-21 15:40:02 at Microsoft.AspNetCore.Mvc.Infrastructure.ActionMethodExecutor.TaskOfIActionResultExecutor.Execute(ActionContext actionContext, IActionResultTypeMapper mapper, ObjectMethodExecutor executor, Object controller, Object[] arguments) 2023-07-21 15:40:02 at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.g__Awaited|12_0(ControllerActionInvoker invoker, ValueTask`1 actionResultValueTask) 2023-07-21 15:40:02 at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.g__Awaited|10_0(ControllerActionInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted) 2023-07-21 15:40:02 at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Rethrow(ActionExecutedContextSealed context) 2023-07-21 15:40:02 at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Next(State& next, Scope& scope, Object& state, Boolean& isCompleted) 2023-07-21 15:40:02 at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.g__Awaited|13_0(ControllerActionInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted) 2023-07-21 15:40:02 at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.g__Awaited|25_0(ResourceInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted) 2023-07-21 15:40:02 at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.Rethrow(ResourceExecutedContextSealed context) 2023-07-21 15:40:02 at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.Next(State& next, Scope& scope, Object& state, Boolean& isCompleted) 2023-07-21 15:40:02 at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.g__Awaited|20_0(ResourceInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted) 2023-07-21 15:40:02 at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.g__Awaited|17_0(ResourceInvoker invoker, Task task, IDisposable scope) 2023-07-21 15:40:02 at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.g__Awaited|17_0(ResourceInvoker invoker, Task task, IDisposable scope) 2023-07-21 15:40:02 at Microsoft.AspNetCore.Routing.EndpointMiddleware.g__AwaitRequestTask|6_0(Endpoint endpoint, Task requestTask, ILogger logger) 2023-07-21 15:40:02 at Microsoft.AspNetCore.Authorization.AuthorizationMiddleware.Invoke(HttpContext context) 2023-07-21 15:40:02 at Microsoft.AspNetCore.Diagnostics.ExceptionHandlerMiddlewareImpl.g__Awaited|8_0(ExceptionHandlerMiddlewareImpl middleware, HttpContext context, Task task)

~eric-pierce 1 year, 1 day ago

~peanutlasko weird I was able to look up the account I seeded in the DB using the bmu interface without that error - but frankly there isn't much value in using the interface. You can just look up users in the mastodon UI and follow them from there, so long as that's working you should be in good shape.

~peanutlasko 1 year, 1 day ago

Right, I can find NPR just fine too. I was under the impression though you should be able to search ANY twitter handle (even if it's not in the database yet), correct?

In Birdsitelive that was the case. Then you could get the Mastodon handle, copy/paste it into Mastodon.

~peanutlasko 1 year, 1 day ago*

Wait, I think i understand now - it's trying to find someone who follows this handle already (@MKBHD@myselfhosteddomain.com) and fails (because nobody does)

If I add it via Mastodon FIRST, then search on Birdmakeup, it's working. So the error only happens whenever I'm searching and haven't added them prior in Mastodon. Makes sense actually.

~eric-pierce 1 year, 1 day ago

~peanutlasko gotcha, glad it sounds like it's working as intended now - clearly the setup issue raised wasn't addressed by ~cloutier for new installs, so unfortunately the manual insert of data is needed, and the table structures have changed since I put together the sample values a couple of months ago

~peanutlasko 1 year, 1 day ago

Now my next question is, lol - how can I mass add all my twitter followers into this. I assume with a DB insert ?

~eric-pierce 1 year, 1 day ago

~peanutlasko as far as I know the only way is to follow each account from your mastodon instance, as both BMU and your mastodon account have to recognize and register the follow

~rursache 1 year, 1 day ago*

what i did to batch follow around 250 accounts was to create a txt file with each twitter username on a separate line like:

@username@birdmakeup.instance.tld
@username2@birdmakeup.instance.tld
….

and then go to mastodon account settings and use the “import following accounts” option + selecting the file.

took around 20mins but it all went great.

~rursache a year ago

~eric-pierce thanks for providing updated initial db imports, i've also updated my "guide" so new users will struggle less: https://github.com/rursache/bird.makeup/issues/3

~retiolus 9 months ago

Hi, I'm having the same issue, I've imported the two lines into the database but even so, it doesn't work. Even the web interface finds the Twitter user, Mastodon doesn't.

birdmakeup=# SELECT * FROM followers;
SELECT * FROM twitter_users;
 id | followings |   acct   |   host   |      inboxroute       | sharedinboxroute |             actorid             | postingerrorcount
----+------------+----------+----------+-----------------------+------------------+---------------------------------+-------------------
  3 | {1}        | retiolus | mamot.fr | /users/retiolus/inbox | /inbox           | https://mamot.fr/users/retiolus |                 0
(1 row)

 id | acct |  lasttweetpostedid  |        lastsync        | fetchingerrorcount | twitteruserid | fediverseaccount | statusescount | extradata
----+------+---------------------+------------------------+--------------------+---------------+------------------+---------------+-----------
  3 | npr  | 1646138106595139584 | 2023-07-21 14:54:07.16 |                  0 |       5392522 |                  |               |
(1 row)

~eric-pierce 8 months ago

~retiolus the version hosted here on sr.ht hasn't worked for me for awhile, I've been using the version hosted on github here: https://github.com/rursache/bird.makeup which has some modifications and is slower to roll out changes, but seemingly is more stable. Thanks to ~rursache for working on this

~eric-pierce 8 months ago

One note that the version on github uses database version 3.1 not 3.2 as the sr.ht version does. To get the github version working you only need to edit the entry in the db_version table to be 3, 1 instead of 3, 2, and then drop the "extradata" columns from the twitterusers and instagramusers tables.

Register here or Log in to comment, or comment via email.